Re: [Nmc-wg] Circuit monitoring - next draft of MDM proposal
Hi, I'm sending updated document of MDM circuit monitoring. Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3). Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities: http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN) waiting for your comments Cheers, Roman
On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
Hi,
I'm sending updated document of MDM circuit monitoring.
Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3).
Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities:
http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN)
So the proposal uses eventTypes for operations (e.g. http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this services can be used to store measurements"). My recollection is that the goal was to move away from using eventTypes as operations; they'd just be used to describe functions. To do this, we'd need some way of including the supported messages in the elements registered up by the services. The clients could then look for things like "the services that support MetadataStoreRequest and utilization data". I'm not positive the best way forward on this. As to the parameter-based eventTypes, I think we need to move away from those if, for no other reason, than to ensure that clients can uniformly query the LS without having to worry about what form the eventType will take. Having multiple ways, especially if different software packages do things differently, is more likely to result in clients that can only work with one type or the other. A few other comments and questions on the document after a first pass. 1. We'll need to think about the specifics of how to generate the segment identifiers when there's a OSCARS/AutoBAHN bridging. When this happens, there are 2 very different identifier schemes, and it's not clear how an agent would go about generating the circuit descriptor since they need to know how other domains are going to generate their segment identifiers. 2. In the segment and circuit descriptors, there are two different relationship names used: 'connects' and 'serialcompound'. In both cases, the relationship is describing the constituent elements that make up the link (links in the case of the circuit descriptor, and ports in the case of the segment descriptor). Since they're describing the same conceptual relationship, having the same name would be good. 3. It looks like the SIP agent is registering the segment ID. I'm curious why it's being registered there. 4. All of the XQuery requests to the hLS are of the form "what services contain eventType X?". This type of request can be handled using the summary requests which has the side benefit of not embedding more XQuery into protocol. 5. Relatedly, I'd like to specify that XPath queries be the only ones used the TS. This way there's a hope of moving away from the XQuery-based protocols, and their dependency on an XML Database. Since XPath can be converted to SQL requests, that'd be much easier to transition to something else. It'd probably also be good to specify exactly the subset of XPath to be supported so that we don't hit the XQuery issue where you are required to have a full-blown XPath implementation to implement the protocol. To keep the XPath syntax simplified, it'd probably be good to eschew the use of functions (at least in the beginning). e.g.: using /*:link[@id="XYZ"] or we could strip off the new for the namespace and have /link[@id="XYZ"] . 6. In the TSQueryResponse example, it looks like far more is returned than I'd expect given the query. Was this just a copy-and-paste instance? 7. The TSAddRequest wraps the individual network elements in an nml:topology element. In the TSQueryResponse response, the individual elements are wrapped in a 'datum'. It'd probably make sense to have the TSAddRequest wrap the individual elements in a 'datum' as well. 8. What kinds of additional info are you thinking about in 9.2.1.9? Is this stuff that is expected to be retrievable by the client, or is it stuff that the MA is going to strip out? 9. I'm curious what kinds of subjects are envisioned for the SIP to MP messages. 10. In 9.2.2.1, the client does a request for every service containing anything on pionier.net. That could easily become non-scalable. It might make sense to qualify the request a bit (e.g. by including eventTypes of interest): <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"><summary:subject> <summary:subject> <nml:domain id="pionier.net"/> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> 11. Also, Jason noticed that the LS response has a datum element that it shouldn't: <nmwg:message> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"> <summary:subject> <nml:domain id="psu.edu" /> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668"> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="3b010db87b462c778bd017447fcc762c"> <perfsonar:subject xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/"> <psservice:service xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <psservice:serviceName>Lookup Service</psservice:serviceName> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint> <psservice:serviceType>hLS</psservice:serviceType> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn State in University Park, PA USA</psservice:serviceDescription> </psservice:service> </perfsonar:subject> </nmwg:metadata> </nmwg:data> </nmwg:message> Cheers, Aaron
W dniu 2011-03-28 14:23, Aaron Brown pisze:
On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
Hi,
I'm sending updated document of MDM circuit monitoring.
Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3).
Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities:
http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN)
So the proposal uses eventTypes for operations (e.g. http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this services can be used to store measurements"). My recollection is that the goal was to move away from using eventTypes as operations; they'd just be used to describe functions. To do this, we'd need some way of including the supported messages in the elements registered up by the services. The clients could then look for things like "the services that support MetadataStoreRequest and utilization data". I'm not positive the best way forward on this.
I would not use message types because they also might be misleading like service types. I like the idea of registering the operations that are supported by a service. Changing a bit your example: "the services that support store operation of utilization data".
As to the parameter-based eventTypes, I think we need to move away from those if, for no other reason, than to ensure that clients can uniformly query the LS without having to worry about what form the eventType will take. Having multiple ways, especially if different software packages do things differently, is more likely to result in clients that can only work with one type or the other.
I've used eventType parameters and I knew this wasn't a good solution but I'd like to see your comments on this. The parameter 'operation' would be more suitable. What do you think?
A few other comments and questions on the document after a first pass.
1. We'll need to think about the specifics of how to generate the segment identifiers when there's a OSCARS/AutoBAHN bridging. When this happens, there are 2 very different identifier schemes, and it's not clear how an agent would go about generating the circuit descriptor since they need to know how other domains are going to generate their segment identifiers.
But the GLIF format and its domain part isn't enough? An agent does not have to know what the last part of the GLIF urn means. (I'll forward this comment to the AutoBAHN team)
2. In the segment and circuit descriptors, there are two different relationship names used: 'connects' and 'serialcompound'. In both cases, the relationship is describing the constituent elements that make up the link (links in the case of the circuit descriptor, and ports in the case of the segment descriptor). Since they're describing the same conceptual relationship, having the same name would be good.
Agree. I was thinking about this while writing descriptor examples in the doc but which one to choose? I would take "connects" but this must be agreed by all of us. So are you fine to take "connects"?
3. It looks like the SIP agent is registering the segment ID. I'm curious why it's being registered there.
The SIP contains info about running monitoring sessions and stores it after they are finished to keep the history (detailed info - segment descriptors - can be fetched from the MA(t) or the MA(at)).
4. All of the XQuery requests to the hLS are of the form "what services contain eventType X?". This type of request can be handled using the summary requests which has the side benefit of not embedding more XQuery into protocol.
5. Relatedly, I'd like to specify that XPath queries be the only ones used the TS. This way there's a hope of moving away from the XQuery-based protocols, and their dependency on an XML Database. Since XPath can be converted to SQL requests, that'd be much easier to transition to something else. It'd probably also be good to specify exactly the subset of XPath to be supported so that we don't hit the XQuery issue where you are required to have a full-blown XPath implementation to implement the protocol. To keep the XPath syntax simplified, it'd probably be good to eschew the use of functions (at least in the beginning). e.g.: using /*:link[@id="XYZ"] or we could strip off the new for the namespace and have /link[@id="XYZ"] .
To keep compatibility I'll replace xquery statements with xpaths (we have started doing this).
6. In the TSQueryResponse example, it looks like far more is returned than I'd expect given the query. Was this just a copy-and-paste instance?
Yes, you're right (SIP->MA(t)). There's a short comment in the request why.
7. The TSAddRequest wraps the individual network elements in an nml:topology element. In the TSQueryResponse response, the individual elements are wrapped in a 'datum'. It'd probably make sense to have the TSAddRequest wrap the individual elements in a 'datum' as well.
Fine for me (I didn't do it this way because I followed the example http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-TopologySer... where there's no datum element)
8. What kinds of additional info are you thinking about in 9.2.1.9? Is this stuff that is expected to be retrievable by the client, or is it stuff that the MA is going to strip out?
Additional info in parameters may contain some general info about the reservation and its monitoring (for example: duration) but now I don't have anything specific in my mind.
9. I'm curious what kinds of subjects are envisioned for the SIP to MP messages.
I don't think the MDM release has an MP service (collecting the link status) ready to be used so it will be developed sooner or later. It's difficult now to say how a possible request will look like (not sure I'll be involved in the work on messages for it).
10. In 9.2.2.1, the client does a request for every service containing anything on pionier.net <http://pionier.net>. That could easily become non-scalable. It might make sense to qualify the request a bit (e.g. by including eventTypes of interest):
This request goes to the gLS so only address(es) of hLS(es) of pionier domain will be provided (I don't expect a domain may have many hLS(es)). But of course additional filtering info may be used (depends on what is summarized and sent by the hLS to the gLS).
<nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"><summary:subject> <summary:subject> <nml:domain id="pionier.net <http://pionier.net>"/> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata>
11. Also, Jason noticed that the LS response has a datum element that it shouldn't:
<nmwg:message> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"> <summary:subject> <nml:domain id="psu.edu <http://psu.edu>" /> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668"> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="3b010db87b462c778bd017447fcc762c"> <perfsonar:subject xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/"> <psservice:service xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <psservice:serviceName>Lookup Service</psservice:serviceName> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint> <psservice:serviceType>hLS</psservice:serviceType> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn State in University Park, PA USA</psservice:serviceDescription> </psservice:service> </perfsonar:subject> </nmwg:metadata> </nmwg:data> </nmwg:message>
I'll update it. thanks, Roman
Cheers, Aaron
Hey Roman, Comments inline. Cheers, Aaron On Mar 29, 2011, at 8:31 AM, Roman Łapacz wrote:
W dniu 2011-03-28 14:23, Aaron Brown pisze:
On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
Hi,
I'm sending updated document of MDM circuit monitoring.
Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3).
Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities:
http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN)
So the proposal uses eventTypes for operations (e.g. http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this services can be used to store measurements"). My recollection is that the goal was to move away from using eventTypes as operations; they'd just be used to describe functions. To do this, we'd need some way of including the supported messages in the elements registered up by the services. The clients could then look for things like "the services that support MetadataStoreRequest and utilization data". I'm not positive the best way forward on this.
I would not use message types because they also might be misleading like service types. I like the idea of registering the operations that are supported by a service. Changing a bit your example: "the services that support store operation of utilization data".
I guess the open question here is what meaning service type (or supported operation) is meant to convey. Is there an instance where two different services would support the same data types and the same messages, but have different semantics?
As to the parameter-based eventTypes, I think we need to move away from those if, for no other reason, than to ensure that clients can uniformly query the LS without having to worry about what form the eventType will take. Having multiple ways, especially if different software packages do things differently, is more likely to result in clients that can only work with one type or the other.
I've used eventType parameters and I knew this wasn't a good solution but I'd like to see your comments on this. The parameter 'operation' would be more suitable. What do you think?
A few other comments and questions on the document after a first pass.
1. We'll need to think about the specifics of how to generate the segment identifiers when there's a OSCARS/AutoBAHN bridging. When this happens, there are 2 very different identifier schemes, and it's not clear how an agent would go about generating the circuit descriptor since they need to know how other domains are going to generate their segment identifiers.
But the GLIF format and its domain part isn't enough? An agent does not have to know what the last part of the GLIF urn means. (I'll forward this comment to the AutoBAHN team)
In the model, the starting domain creates the circuit descriptor. However, to do that, that domain needs to know how the other domains are going to set the identifiers for their segment descriptors. There are two ways to do that: 1) have a programmatic way that each domain uses to create the segment id using information that is available to all domains 2) have the starting domain contact each of the other domains and request the segment identifiers I'd prefer #1 if possible since it makes it substantially easier to deploy. However, for #1, there's not a good way, if OSCARS is the starting domain, to figure out how an AutoBAHN domain is going to name its segments since all OSCARS will have is a GRI (not compatible with AutoBAHN's id structure), and the domain names.
2. In the segment and circuit descriptors, there are two different relationship names used: 'connects' and 'serialcompound'. In both cases, the relationship is describing the constituent elements that make up the link (links in the case of the circuit descriptor, and ports in the case of the segment descriptor). Since they're describing the same conceptual relationship, having the same name would be good.
Agree. I was thinking about this while writing descriptor examples in the doc but which one to choose? I would take "connects" but this must be agreed by all of us. So are you fine to take "connects"?
I'd just been using "over", but would be fine with "connects".
3. It looks like the SIP agent is registering the segment ID. I'm curious why it's being registered there.
The SIP contains info about running monitoring sessions and stores it after they are finished to keep the history (detailed info - segment descriptors - can be fetched from the MA(t) or the MA(at)).
4. All of the XQuery requests to the hLS are of the form "what services contain eventType X?". This type of request can be handled using the summary requests which has the side benefit of not embedding more XQuery into protocol.
5. Relatedly, I'd like to specify that XPath queries be the only ones used the TS. This way there's a hope of moving away from the XQuery-based protocols, and their dependency on an XML Database. Since XPath can be converted to SQL requests, that'd be much easier to transition to something else. It'd probably also be good to specify exactly the subset of XPath to be supported so that we don't hit the XQuery issue where you are required to have a full-blown XPath implementation to implement the protocol. To keep the XPath syntax simplified, it'd probably be good to eschew the use of functions (at least in the beginning). e.g.: using /*:link[@id="XYZ"] or we could strip off the new for the namespace and have /link[@id="XYZ"] .
To keep compatibility I'll replace xquery statements with xpaths (we have started doing this).
6. In the TSQueryResponse example, it looks like far more is returned than I'd expect given the query. Was this just a copy-and-paste instance?
Yes, you're right (SIP->MA(t)). There's a short comment in the request why.
7. The TSAddRequest wraps the individual network elements in an nml:topology element. In the TSQueryResponse response, the individual elements are wrapped in a 'datum'. It'd probably make sense to have the TSAddRequest wrap the individual elements in a 'datum' as well.
Fine for me (I didn't do it this way because I followed the example http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-TopologySer... where there's no datum element)
Yeah, I figured as such. I'd just like to change the semantics to be more similar to other services.
8. What kinds of additional info are you thinking about in 9.2.1.9? Is this stuff that is expected to be retrievable by the client, or is it stuff that the MA is going to strip out?
Additional info in parameters may contain some general info about the reservation and its monitoring (for example: duration) but now I don't have anything specific in my mind.
9. I'm curious what kinds of subjects are envisioned for the SIP to MP messages.
I don't think the MDM release has an MP service (collecting the link status) ready to be used so it will be developed sooner or later. It's difficult now to say how a possible request will look like (not sure I'll be involved in the work on messages for it).
10. In 9.2.2.1, the client does a request for every service containing anything on pionier.net. That could easily become non-scalable. It might make sense to qualify the request a bit (e.g. by including eventTypes of interest):
This request goes to the gLS so only address(es) of hLS(es) of pionier domain will be provided (I don't expect a domain may have many hLS(es)). But of course additional filtering info may be used (depends on what is summarized and sent by the hLS to the gLS).
Well, the result would be any hLS that has any data on the pionier domain. So if there are other domains out there that running PingER, HADES, owamp or bwctl tests against pionier hosts, they'd be returned as well. On the flipside, I'm not sure adding the eventType would help this situation since many of those domains could also have utilization data (which may or may not be applicable to the pionier domain), so a search for utilization data and pionier could grab the same set of domains. I'm wondering if the LS summarization should account for the relationships between the data and the domains. The semantics of the summary query to the gLS would be "give me the hLSes that know about services with utilization data about internet2.edu", as opposed to, "give me the hLSes that know about services with data about internet2.edu, and services with utilization data).
<nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"><summary:subject> <summary:subject> <nml:domain id="pionier.net"/> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata>
11. Also, Jason noticed that the LS response has a datum element that it shouldn't:
<nmwg:message> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"> <summary:subject> <nml:domain id="psu.edu" /> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668"> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="3b010db87b462c778bd017447fcc762c"> <perfsonar:subject xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/"> <psservice:service xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <psservice:serviceName>Lookup Service</psservice:serviceName> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint> <psservice:serviceType>hLS</psservice:serviceType> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn State in University Park, PA USA</psservice:serviceDescription> </psservice:service> </perfsonar:subject> </nmwg:metadata> </nmwg:data> </nmwg:message>
I'll update it.
thanks, Roman
Cheers, Aaron
W dniu 2011-03-29 17:57, Aaron Brown pisze:
Hey Roman,
Comments inline.
Cheers, Aaron
On Mar 29, 2011, at 8:31 AM, Roman Łapacz wrote:
W dniu 2011-03-28 14:23, Aaron Brown pisze:
On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
Hi,
I'm sending updated document of MDM circuit monitoring.
Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3).
Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities:
http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN)
So the proposal uses eventTypes for operations (e.g. http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this services can be used to store measurements"). My recollection is that the goal was to move away from using eventTypes as operations; they'd just be used to describe functions. To do this, we'd need some way of including the supported messages in the elements registered up by the services. The clients could then look for things like "the services that support MetadataStoreRequest and utilization data". I'm not positive the best way forward on this.
I would not use message types because they also might be misleading like service types. I like the idea of registering the operations that are supported by a service. Changing a bit your example: "the services that support store operation of utilization data".
I guess the open question here is what meaning service type (or supported operation) is meant to convey. Is there an instance where two different services would support the same data types and the same messages, but have different semantics?
I don't think there's such an instance but it's possible to have it in future. Jason, what do you think about 'operation' parameters or use of message types? (For the time being I'll replace eventType parameters with operation parameters and later update it when we have something better. In fact I have to do this because topology services of two pS implantation have different service types - "TS" and "MA").
As to the parameter-based eventTypes, I think we need to move away from those if, for no other reason, than to ensure that clients can uniformly query the LS without having to worry about what form the eventType will take. Having multiple ways, especially if different software packages do things differently, is more likely to result in clients that can only work with one type or the other.
I've used eventType parameters and I knew this wasn't a good solution but I'd like to see your comments on this. The parameter 'operation' would be more suitable. What do you think?
A few other comments and questions on the document after a first pass.
1. We'll need to think about the specifics of how to generate the segment identifiers when there's a OSCARS/AutoBAHN bridging. When this happens, there are 2 very different identifier schemes, and it's not clear how an agent would go about generating the circuit descriptor since they need to know how other domains are going to generate their segment identifiers.
But the GLIF format and its domain part isn't enough? An agent does not have to know what the last part of the GLIF urn means. (I'll forward this comment to the AutoBAHN team)
In the model, the starting domain creates the circuit descriptor. However, to do that, that domain needs to know how the other domains are going to set the identifiers for their segment descriptors. There are two ways to do that:
1) have a programmatic way that each domain uses to create the segment id using information that is available to all domains 2) have the starting domain contact each of the other domains and request the segment identifiers
I'd prefer #1 if possible since it makes it substantially easier to deploy. However, for #1, there's not a good way, if OSCARS is the starting domain, to figure out how an AutoBAHN domain is going to name its segments since all OSCARS will have is a GRI (not compatible with AutoBAHN's id structure), and the domain names.
I'm afraid that #2 will have to be used in such case. I talked to the AutoBAHN guy and he promised to comment this soon. My first impression is that #1 will require some extended communication between those two systems and thus some major changes in the existing implementations.
2. In the segment and circuit descriptors, there are two different relationship names used: 'connects' and 'serialcompound'. In both cases, the relationship is describing the constituent elements that make up the link (links in the case of the circuit descriptor, and ports in the case of the segment descriptor). Since they're describing the same conceptual relationship, having the same name would be good.
Agree. I was thinking about this while writing descriptor examples in the doc but which one to choose? I would take "connects" but this must be agreed by all of us. So are you fine to take "connects"?
I'd just been using "over", but would be fine with "connects".
Let's stay with "connects" (I remember Freek's concern that "over" refers more to multi-layering).
3. It looks like the SIP agent is registering the segment ID. I'm curious why it's being registered there.
The SIP contains info about running monitoring sessions and stores it after they are finished to keep the history (detailed info - segment descriptors - can be fetched from the MA(t) or the MA(at)).
4. All of the XQuery requests to the hLS are of the form "what services contain eventType X?". This type of request can be handled using the summary requests which has the side benefit of not embedding more XQuery into protocol.
5. Relatedly, I'd like to specify that XPath queries be the only ones used the TS. This way there's a hope of moving away from the XQuery-based protocols, and their dependency on an XML Database. Since XPath can be converted to SQL requests, that'd be much easier to transition to something else. It'd probably also be good to specify exactly the subset of XPath to be supported so that we don't hit the XQuery issue where you are required to have a full-blown XPath implementation to implement the protocol. To keep the XPath syntax simplified, it'd probably be good to eschew the use of functions (at least in the beginning). e.g.: using /*:link[@id="XYZ"] or we could strip off the new for the namespace and have /link[@id="XYZ"] .
To keep compatibility I'll replace xquery statements with xpaths (we have started doing this).
6. In the TSQueryResponse example, it looks like far more is returned than I'd expect given the query. Was this just a copy-and-paste instance?
Yes, you're right (SIP->MA(t)). There's a short comment in the request why.
7. The TSAddRequest wraps the individual network elements in an nml:topology element. In the TSQueryResponse response, the individual elements are wrapped in a 'datum'. It'd probably make sense to have the TSAddRequest wrap the individual elements in a 'datum' as well.
Fine for me (I didn't do it this way because I followed the example http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-TopologySer... where there's no datum element)
Yeah, I figured as such. I'd just like to change the semantics to be more similar to other services.
8. What kinds of additional info are you thinking about in 9.2.1.9? Is this stuff that is expected to be retrievable by the client, or is it stuff that the MA is going to strip out?
Additional info in parameters may contain some general info about the reservation and its monitoring (for example: duration) but now I don't have anything specific in my mind.
9. I'm curious what kinds of subjects are envisioned for the SIP to MP messages.
I don't think the MDM release has an MP service (collecting the link status) ready to be used so it will be developed sooner or later. It's difficult now to say how a possible request will look like (not sure I'll be involved in the work on messages for it).
10. In 9.2.2.1, the client does a request for every service containing anything on pionier.net <http://pionier.net/>. That could easily become non-scalable. It might make sense to qualify the request a bit (e.g. by including eventTypes of interest):
This request goes to the gLS so only address(es) of hLS(es) of pionier domain will be provided (I don't expect a domain may have many hLS(es)). But of course additional filtering info may be used (depends on what is summarized and sent by the hLS to the gLS).
Well, the result would be any hLS that has any data on the pionier domain. So if there are other domains out there that running PingER, HADES, owamp or bwctl tests against pionier hosts, they'd be returned as well. On the flipside, I'm not sure adding the eventType would help this situation since many of those domains could also have utilization data (which may or may not be applicable to the pionier domain), so a search for utilization data and pionier could grab the same set of domains.
I'm thinking about this differently, the gLS would response with the hLS(es) that are deployed in the pionier domain. The hLS would register information that contains (meta)data from specific domain(s). So, for example: "give me the hLSes deployed in/representing pionier.net that know services with utilization data". Cheers, Roman
I'm wondering if the LS summarization should account for the relationships between the data and the domains. The semantics of the summary query to the gLS would be "give me the hLSes that know about services with utilization data about internet2.edu <http://internet2.edu>", as opposed to, "give me the hLSes that know about services with data about internet2.edu <http://internet2.edu>, and services with utilization data).
<nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"><summary:subject> <summary:subject> <nml:domain id="pionier.net <http://pionier.net/>"/> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata>
11. Also, Jason noticed that the LS response has a datum element that it shouldn't:
<nmwg:message> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"> <summary:subject> <nml:domain id="psu.edu <http://psu.edu/>" /> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668"> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="3b010db87b462c778bd017447fcc762c"> <perfsonar:subject xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/"> <psservice:service xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <psservice:serviceName>Lookup Service</psservice:serviceName> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint> <psservice:serviceType>hLS</psservice:serviceType> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn State in University Park, PA USA</psservice:serviceDescription> </psservice:service> </perfsonar:subject> </nmwg:metadata> </nmwg:data> </nmwg:message>
I'll update it.
thanks, Roman
Cheers, Aaron
Hi Roman/All; Comments inline: On 3/30/11 8:04 AM, Roman Łapacz wrote:
W dniu 2011-03-29 17:57, Aaron Brown pisze:
Hey Roman,
Comments inline.
Cheers, Aaron
On Mar 29, 2011, at 8:31 AM, Roman Łapacz wrote:
W dniu 2011-03-28 14:23, Aaron Brown pisze:
On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
Hi,
I'm sending updated document of MDM circuit monitoring.
Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3).
Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities:
http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN)
So the proposal uses eventTypes for operations (e.g. http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this services can be used to store measurements"). My recollection is that the goal was to move away from using eventTypes as operations; they'd just be used to describe functions. To do this, we'd need some way of including the supported messages in the elements registered up by the services. The clients could then look for things like "the services that support MetadataStoreRequest and utilization data". I'm not positive the best way forward on this.
I would not use message types because they also might be misleading like service types. I like the idea of registering the operations that are supported by a service. Changing a bit your example: "the services that support store operation of utilization data".
I guess the open question here is what meaning service type (or supported operation) is meant to convey. Is there an instance where two different services would support the same data types and the same messages, but have different semantics?
I don't think there's such an instance but it's possible to have it in future. Jason, what do you think about 'operation' parameters or use of message types? (For the time being I'll replace eventType parameters with operation parameters and later update it when we have something better. In fact I have to do this because topology services of two pS implantation have different service types - "TS" and "MA").
I have to admit I am not following this proposal very well, why do you feel it is necessary to do this? Topology data is topology data, no matter who is storing it. I am not really sure why it's important to be able to identify the type of service directly via the Lookup Service in this case. If you can go into more detail about why this is necessary at the information services level perhaps that would be helpful. If you need to identify what type the service really is you can always ask it directly, after the Lookup Service tells you that it contains topology. This is just one extra step after all, and avoids trying to impart yet another meaning on the eventType. I would rather not overload the eventType to mean both "type of data" and "service capability" unless there is a very compelling reason to do so. You mention also using a "parameter", how would that work? Would this be similar to how we currently use the "keyword" construct? Note that if this were the case the summarization algorithm and discovery APIs would need to change, this is not impossible of course, but I would like to know the entire reason before forcing a global flag day to implement this.
As to the parameter-based eventTypes, I think we need to move away from those if, for no other reason, than to ensure that clients can uniformly query the LS without having to worry about what form the eventType will take. Having multiple ways, especially if different software packages do things differently, is more likely to result in clients that can only work with one type or the other.
I've used eventType parameters and I knew this wasn't a good solution but I'd like to see your comments on this. The parameter 'operation' would be more suitable. What do you think?
A few other comments and questions on the document after a first pass.
1. We'll need to think about the specifics of how to generate the segment identifiers when there's a OSCARS/AutoBAHN bridging. When this happens, there are 2 very different identifier schemes, and it's not clear how an agent would go about generating the circuit descriptor since they need to know how other domains are going to generate their segment identifiers.
But the GLIF format and its domain part isn't enough? An agent does not have to know what the last part of the GLIF urn means. (I'll forward this comment to the AutoBAHN team)
In the model, the starting domain creates the circuit descriptor. However, to do that, that domain needs to know how the other domains are going to set the identifiers for their segment descriptors. There are two ways to do that:
1) have a programmatic way that each domain uses to create the segment id using information that is available to all domains 2) have the starting domain contact each of the other domains and request the segment identifiers
I'd prefer #1 if possible since it makes it substantially easier to deploy. However, for #1, there's not a good way, if OSCARS is the starting domain, to figure out how an AutoBAHN domain is going to name its segments since all OSCARS will have is a GRI (not compatible with AutoBAHN's id structure), and the domain names.
I'm afraid that #2 will have to be used in such case. I talked to the AutoBAHN guy and he promised to comment this soon. My first impression is that #1 will require some extended communication between those two systems and thus some major changes in the existing implementations.
2. In the segment and circuit descriptors, there are two different relationship names used: 'connects' and 'serialcompound'. In both cases, the relationship is describing the constituent elements that make up the link (links in the case of the circuit descriptor, and ports in the case of the segment descriptor). Since they're describing the same conceptual relationship, having the same name would be good.
Agree. I was thinking about this while writing descriptor examples in the doc but which one to choose? I would take "connects" but this must be agreed by all of us. So are you fine to take "connects"?
I'd just been using "over", but would be fine with "connects".
Let's stay with "connects" (I remember Freek's concern that "over" refers more to multi-layering).
3. It looks like the SIP agent is registering the segment ID. I'm curious why it's being registered there.
The SIP contains info about running monitoring sessions and stores it after they are finished to keep the history (detailed info - segment descriptors - can be fetched from the MA(t) or the MA(at)).
4. All of the XQuery requests to the hLS are of the form "what services contain eventType X?". This type of request can be handled using the summary requests which has the side benefit of not embedding more XQuery into protocol.
5. Relatedly, I'd like to specify that XPath queries be the only ones used the TS. This way there's a hope of moving away from the XQuery-based protocols, and their dependency on an XML Database. Since XPath can be converted to SQL requests, that'd be much easier to transition to something else. It'd probably also be good to specify exactly the subset of XPath to be supported so that we don't hit the XQuery issue where you are required to have a full-blown XPath implementation to implement the protocol. To keep the XPath syntax simplified, it'd probably be good to eschew the use of functions (at least in the beginning). e.g.: using /*:link[@id="XYZ"] or we could strip off the new for the namespace and have /link[@id="XYZ"] .
To keep compatibility I'll replace xquery statements with xpaths (we have started doing this).
6. In the TSQueryResponse example, it looks like far more is returned than I'd expect given the query. Was this just a copy-and-paste instance?
Yes, you're right (SIP->MA(t)). There's a short comment in the request why.
7. The TSAddRequest wraps the individual network elements in an nml:topology element. In the TSQueryResponse response, the individual elements are wrapped in a 'datum'. It'd probably make sense to have the TSAddRequest wrap the individual elements in a 'datum' as well.
Fine for me (I didn't do it this way because I followed the example http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-TopologySer... where there's no datum element)
Yeah, I figured as such. I'd just like to change the semantics to be more similar to other services.
8. What kinds of additional info are you thinking about in 9.2.1.9? Is this stuff that is expected to be retrievable by the client, or is it stuff that the MA is going to strip out?
Additional info in parameters may contain some general info about the reservation and its monitoring (for example: duration) but now I don't have anything specific in my mind.
9. I'm curious what kinds of subjects are envisioned for the SIP to MP messages.
I don't think the MDM release has an MP service (collecting the link status) ready to be used so it will be developed sooner or later. It's difficult now to say how a possible request will look like (not sure I'll be involved in the work on messages for it).
10. In 9.2.2.1, the client does a request for every service containing anything on pionier.net <http://pionier.net/>. That could easily become non-scalable. It might make sense to qualify the request a bit (e.g. by including eventTypes of interest):
This request goes to the gLS so only address(es) of hLS(es) of pionier domain will be provided (I don't expect a domain may have many hLS(es)). But of course additional filtering info may be used (depends on what is summarized and sent by the hLS to the gLS).
Well, the result would be any hLS that has any data on the pionier domain. So if there are other domains out there that running PingER, HADES, owamp or bwctl tests against pionier hosts, they'd be returned as well. On the flipside, I'm not sure adding the eventType would help this situation since many of those domains could also have utilization data (which may or may not be applicable to the pionier domain), so a search for utilization data and pionier could grab the same set of domains.
I'm thinking about this differently, the gLS would response with the hLS(es) that are deployed in the pionier domain. The hLS would register information that contains (meta)data from specific domain(s). So, for example: "give me the hLSes deployed in/representing pionier.net that know services with utilization data".
The queries don't really work that way, you can't ask a question such as "tell me who the authoritative hLS is for domain x". Remember this is a federated model, anyone can set up an hLS at any time in a given domain (without the knowledge of someone else who as already set one up) and claim it is authoritative. You could conceivably also do this in an unrelated domain. The queries are designed to revolve around the data itself instead of some notion of administrative knowledge. You search for data that is related to domain x, and you receive many answers (e.g. data that originated there and has an end in another domain, or the reverse). Aaron's (and my) concern is that doing a query for just a domain qualifier is the same thing as dong a DNS zone transfer - it dumps lots and lots of information and is very resource consuming. There is nothing that stops you from doing this of course, its just not something a kind user should do. I believe it is possible for a more specific query (e.g. give me topology data for domain x, or give me topology data for domain x and domain y) to be used instead. Is this fair? Thanks; -jason
Cheers, Roman
I'm wondering if the LS summarization should account for the relationships between the data and the domains. The semantics of the summary query to the gLS would be "give me the hLSes that know about services with utilization data about internet2.edu <http://internet2.edu>", as opposed to, "give me the hLSes that know about services with data about internet2.edu <http://internet2.edu>, and services with utilization data).
<nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"><summary:subject> <summary:subject> <nml:domain id="pionier.net <http://pionier.net/>"/> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata>
11. Also, Jason noticed that the LS response has a datum element that it shouldn't:
<nmwg:message> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"> <summary:subject> <nml:domain id="psu.edu <http://psu.edu/>" /> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668"> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="3b010db87b462c778bd017447fcc762c"> <perfsonar:subject xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/"> <psservice:service xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <psservice:serviceName>Lookup Service</psservice:serviceName> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint> <psservice:serviceType>hLS</psservice:serviceType> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn State in University Park, PA USA</psservice:serviceDescription> </psservice:service> </perfsonar:subject> </nmwg:metadata> </nmwg:data> </nmwg:message>
I'll update it.
thanks, Roman
Cheers, Aaron
Hi Aaron/All, I'm sending an example of extended lookup information (this is what we discussed yesterday; two new things: eventType and message types in the lookup info): <?xml version="1.0" encoding="UTF-8"?> <nmwg:message id="msg1" type="LSRegisterRequest" xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/" xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <nmwg:metadata id="meta1"> <perfsonar:subject id="1"> <psservice:service id="mat_1"> <psservice:serviceName>Measurement Archive for intra-domain topology</psservice:serviceName> <psservice:accessPoint>http://host:8080/services/ma_t</psservice:accessPoint> <psservice:serviceType>MA</psservice:serviceType> <psservice:organization>PIONIER</psservice:organization> </psservice:service> </perfsonar:subject> <!-- eventType indicates what kind of data the service supports --> <nmwg:eventType>http://ggf.org/ns/nmwg/topology/201103</nmwg:eventType> <psservice:parameters> <nmwg:parameter name="supportedMessage">TSAddRequest</nmwg:parameter> <nmwg:parameter name="supportedMessage">TSQueryRequest</nmwg:parameter> </psservice:parameters> </nmwg:metadata> <!-- There is no data element(s) with metadata information because --> <!-- at the beginning (when the service starts) MA(t) does not store any data --> <nmwg:data id="data1" metadataIdRef="meta1"/> </nmwg:message> The problem I see is that the MA and the MP may use the same message type (SetupDataRequest) for different actions. In case of the MA it is fetching data. For the MP it is executing a test or collecting data. I don't think we've had a separate message type defined for the MP and if I'm not wrong SetupDataRequest was used (until it is not clarified I prefer to have 'operation' parameter). Cheers, Roman W dniu 2011-03-29 17:57, Aaron Brown pisze:
Hey Roman,
Comments inline.
Cheers, Aaron
On Mar 29, 2011, at 8:31 AM, Roman Łapacz wrote:
W dniu 2011-03-28 14:23, Aaron Brown pisze:
On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
Hi,
I'm sending updated document of MDM circuit monitoring.
Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3).
Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities:
http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN)
So the proposal uses eventTypes for operations (e.g. http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this services can be used to store measurements"). My recollection is that the goal was to move away from using eventTypes as operations; they'd just be used to describe functions. To do this, we'd need some way of including the supported messages in the elements registered up by the services. The clients could then look for things like "the services that support MetadataStoreRequest and utilization data". I'm not positive the best way forward on this.
I would not use message types because they also might be misleading like service types. I like the idea of registering the operations that are supported by a service. Changing a bit your example: "the services that support store operation of utilization data".
I guess the open question here is what meaning service type (or supported operation) is meant to convey. Is there an instance where two different services would support the same data types and the same messages, but have different semantics?
As to the parameter-based eventTypes, I think we need to move away from those if, for no other reason, than to ensure that clients can uniformly query the LS without having to worry about what form the eventType will take. Having multiple ways, especially if different software packages do things differently, is more likely to result in clients that can only work with one type or the other.
I've used eventType parameters and I knew this wasn't a good solution but I'd like to see your comments on this. The parameter 'operation' would be more suitable. What do you think?
A few other comments and questions on the document after a first pass.
1. We'll need to think about the specifics of how to generate the segment identifiers when there's a OSCARS/AutoBAHN bridging. When this happens, there are 2 very different identifier schemes, and it's not clear how an agent would go about generating the circuit descriptor since they need to know how other domains are going to generate their segment identifiers.
But the GLIF format and its domain part isn't enough? An agent does not have to know what the last part of the GLIF urn means. (I'll forward this comment to the AutoBAHN team)
In the model, the starting domain creates the circuit descriptor. However, to do that, that domain needs to know how the other domains are going to set the identifiers for their segment descriptors. There are two ways to do that:
1) have a programmatic way that each domain uses to create the segment id using information that is available to all domains 2) have the starting domain contact each of the other domains and request the segment identifiers
I'd prefer #1 if possible since it makes it substantially easier to deploy. However, for #1, there's not a good way, if OSCARS is the starting domain, to figure out how an AutoBAHN domain is going to name its segments since all OSCARS will have is a GRI (not compatible with AutoBAHN's id structure), and the domain names.
2. In the segment and circuit descriptors, there are two different relationship names used: 'connects' and 'serialcompound'. In both cases, the relationship is describing the constituent elements that make up the link (links in the case of the circuit descriptor, and ports in the case of the segment descriptor). Since they're describing the same conceptual relationship, having the same name would be good.
Agree. I was thinking about this while writing descriptor examples in the doc but which one to choose? I would take "connects" but this must be agreed by all of us. So are you fine to take "connects"?
I'd just been using "over", but would be fine with "connects".
3. It looks like the SIP agent is registering the segment ID. I'm curious why it's being registered there.
The SIP contains info about running monitoring sessions and stores it after they are finished to keep the history (detailed info - segment descriptors - can be fetched from the MA(t) or the MA(at)).
4. All of the XQuery requests to the hLS are of the form "what services contain eventType X?". This type of request can be handled using the summary requests which has the side benefit of not embedding more XQuery into protocol.
5. Relatedly, I'd like to specify that XPath queries be the only ones used the TS. This way there's a hope of moving away from the XQuery-based protocols, and their dependency on an XML Database. Since XPath can be converted to SQL requests, that'd be much easier to transition to something else. It'd probably also be good to specify exactly the subset of XPath to be supported so that we don't hit the XQuery issue where you are required to have a full-blown XPath implementation to implement the protocol. To keep the XPath syntax simplified, it'd probably be good to eschew the use of functions (at least in the beginning). e.g.: using /*:link[@id="XYZ"] or we could strip off the new for the namespace and have /link[@id="XYZ"] .
To keep compatibility I'll replace xquery statements with xpaths (we have started doing this).
6. In the TSQueryResponse example, it looks like far more is returned than I'd expect given the query. Was this just a copy-and-paste instance?
Yes, you're right (SIP->MA(t)). There's a short comment in the request why.
7. The TSAddRequest wraps the individual network elements in an nml:topology element. In the TSQueryResponse response, the individual elements are wrapped in a 'datum'. It'd probably make sense to have the TSAddRequest wrap the individual elements in a 'datum' as well.
Fine for me (I didn't do it this way because I followed the example http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-TopologySer... where there's no datum element)
Yeah, I figured as such. I'd just like to change the semantics to be more similar to other services.
8. What kinds of additional info are you thinking about in 9.2.1.9? Is this stuff that is expected to be retrievable by the client, or is it stuff that the MA is going to strip out?
Additional info in parameters may contain some general info about the reservation and its monitoring (for example: duration) but now I don't have anything specific in my mind.
9. I'm curious what kinds of subjects are envisioned for the SIP to MP messages.
I don't think the MDM release has an MP service (collecting the link status) ready to be used so it will be developed sooner or later. It's difficult now to say how a possible request will look like (not sure I'll be involved in the work on messages for it).
10. In 9.2.2.1, the client does a request for every service containing anything on pionier.net <http://pionier.net/>. That could easily become non-scalable. It might make sense to qualify the request a bit (e.g. by including eventTypes of interest):
This request goes to the gLS so only address(es) of hLS(es) of pionier domain will be provided (I don't expect a domain may have many hLS(es)). But of course additional filtering info may be used (depends on what is summarized and sent by the hLS to the gLS).
Well, the result would be any hLS that has any data on the pionier domain. So if there are other domains out there that running PingER, HADES, owamp or bwctl tests against pionier hosts, they'd be returned as well. On the flipside, I'm not sure adding the eventType would help this situation since many of those domains could also have utilization data (which may or may not be applicable to the pionier domain), so a search for utilization data and pionier could grab the same set of domains. I'm wondering if the LS summarization should account for the relationships between the data and the domains. The semantics of the summary query to the gLS would be "give me the hLSes that know about services with utilization data about internet2.edu <http://internet2.edu>", as opposed to, "give me the hLSes that know about services with data about internet2.edu <http://internet2.edu>, and services with utilization data).
<nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"><summary:subject> <summary:subject> <nml:domain id="pionier.net <http://pionier.net/>"/> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata>
11. Also, Jason noticed that the LS response has a datum element that it shouldn't:
<nmwg:message> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"> <summary:subject> <nml:domain id="psu.edu <http://psu.edu/>" /> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668"> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="3b010db87b462c778bd017447fcc762c"> <perfsonar:subject xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/"> <psservice:service xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <psservice:serviceName>Lookup Service</psservice:serviceName> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint> <psservice:serviceType>hLS</psservice:serviceType> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn State in University Park, PA USA</psservice:serviceDescription> </psservice:service> </perfsonar:subject> </nmwg:metadata> </nmwg:data> </nmwg:message>
I'll update it.
thanks, Roman
Cheers, Aaron
Hi Roman/All, I've attached the client messages that I've been using in the early stages of a monitoring client. The general information flow goes: 1. Query the gLS for the hLS that knows about services with topology information about the domain: gls_topology_request.xml 2. Query the hLS for the TS that has topology information about the domain: hls_topology_request.xml 3. Query the TS for all topology in the TS (speeds up when there are numerous bits of information the client needs) topology_request.xml 4. Query the gLS for the hLS that knows about services with utilization information about the domain: gls_utilization_request.xml 5. Query the hLS for the MA that has utilization information about the domain: hls_utilization_request.xml 6. Query the MA for the utilization data about the interfaces from the topology: ma_utilization_request.xml Without a live deployment of all the services, I've had to fudge some of the responses (normally they return no results for the queries). Cheers, Aaron On Apr 1, 2011, at 8:25 AM, Roman Łapacz wrote:
Hi Aaron/All,
I'm sending an example of extended lookup information (this is what we discussed yesterday; two new things: eventType and message types in the lookup info):
<?xml version="1.0" encoding="UTF-8"?> <nmwg:message id="msg1" type="LSRegisterRequest" xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/" xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/">
<nmwg:metadata id="meta1"> <perfsonar:subject id="1"> <psservice:service id="mat_1"> <psservice:serviceName>Measurement Archive for intra-domain topology</psservice:serviceName> <psservice:accessPoint>http://host:8080/services/ma_t</psservice:accessPoint> <psservice:serviceType>MA</psservice:serviceType> <psservice:organization>PIONIER</psservice:organization> </psservice:service> </perfsonar:subject> <!-- eventType indicates what kind of data the service supports --> <nmwg:eventType>http://ggf.org/ns/nmwg/topology/201103</nmwg:eventType> <psservice:parameters> <nmwg:parameter name="supportedMessage">TSAddRequest</nmwg:parameter> <nmwg:parameter name="supportedMessage">TSQueryRequest</nmwg:parameter> </psservice:parameters> </nmwg:metadata>
<!-- There is no data element(s) with metadata information because --> <!-- at the beginning (when the service starts) MA(t) does not store any data --> <nmwg:data id="data1" metadataIdRef="meta1"/>
</nmwg:message>
The problem I see is that the MA and the MP may use the same message type (SetupDataRequest) for different actions. In case of the MA it is fetching data. For the MP it is executing a test or collecting data. I don't think we've had a separate message type defined for the MP and if I'm not wrong SetupDataRequest was used (until it is not clarified I prefer to have 'operation' parameter).
Cheers, Roman
W dniu 2011-03-29 17:57, Aaron Brown pisze:
Hey Roman,
Comments inline.
Cheers, Aaron
On Mar 29, 2011, at 8:31 AM, Roman Łapacz wrote:
W dniu 2011-03-28 14:23, Aaron Brown pisze:
On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
Hi,
I'm sending updated document of MDM circuit monitoring.
Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3).
Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities:
http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN)
So the proposal uses eventTypes for operations (e.g. http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this services can be used to store measurements"). My recollection is that the goal was to move away from using eventTypes as operations; they'd just be used to describe functions. To do this, we'd need some way of including the supported messages in the elements registered up by the services. The clients could then look for things like "the services that support MetadataStoreRequest and utilization data". I'm not positive the best way forward on this.
I would not use message types because they also might be misleading like service types. I like the idea of registering the operations that are supported by a service. Changing a bit your example: "the services that support store operation of utilization data".
I guess the open question here is what meaning service type (or supported operation) is meant to convey. Is there an instance where two different services would support the same data types and the same messages, but have different semantics?
As to the parameter-based eventTypes, I think we need to move away from those if, for no other reason, than to ensure that clients can uniformly query the LS without having to worry about what form the eventType will take. Having multiple ways, especially if different software packages do things differently, is more likely to result in clients that can only work with one type or the other.
I've used eventType parameters and I knew this wasn't a good solution but I'd like to see your comments on this. The parameter 'operation' would be more suitable. What do you think?
A few other comments and questions on the document after a first pass.
1. We'll need to think about the specifics of how to generate the segment identifiers when there's a OSCARS/AutoBAHN bridging. When this happens, there are 2 very different identifier schemes, and it's not clear how an agent would go about generating the circuit descriptor since they need to know how other domains are going to generate their segment identifiers.
But the GLIF format and its domain part isn't enough? An agent does not have to know what the last part of the GLIF urn means. (I'll forward this comment to the AutoBAHN team)
In the model, the starting domain creates the circuit descriptor. However, to do that, that domain needs to know how the other domains are going to set the identifiers for their segment descriptors. There are two ways to do that:
1) have a programmatic way that each domain uses to create the segment id using information that is available to all domains 2) have the starting domain contact each of the other domains and request the segment identifiers
I'd prefer #1 if possible since it makes it substantially easier to deploy. However, for #1, there's not a good way, if OSCARS is the starting domain, to figure out how an AutoBAHN domain is going to name its segments since all OSCARS will have is a GRI (not compatible with AutoBAHN's id structure), and the domain names.
2. In the segment and circuit descriptors, there are two different relationship names used: 'connects' and 'serialcompound'. In both cases, the relationship is describing the constituent elements that make up the link (links in the case of the circuit descriptor, and ports in the case of the segment descriptor). Since they're describing the same conceptual relationship, having the same name would be good.
Agree. I was thinking about this while writing descriptor examples in the doc but which one to choose? I would take "connects" but this must be agreed by all of us. So are you fine to take "connects"?
I'd just been using "over", but would be fine with "connects".
3. It looks like the SIP agent is registering the segment ID. I'm curious why it's being registered there.
The SIP contains info about running monitoring sessions and stores it after they are finished to keep the history (detailed info - segment descriptors - can be fetched from the MA(t) or the MA(at)).
4. All of the XQuery requests to the hLS are of the form "what services contain eventType X?". This type of request can be handled using the summary requests which has the side benefit of not embedding more XQuery into protocol.
5. Relatedly, I'd like to specify that XPath queries be the only ones used the TS. This way there's a hope of moving away from the XQuery-based protocols, and their dependency on an XML Database. Since XPath can be converted to SQL requests, that'd be much easier to transition to something else. It'd probably also be good to specify exactly the subset of XPath to be supported so that we don't hit the XQuery issue where you are required to have a full-blown XPath implementation to implement the protocol. To keep the XPath syntax simplified, it'd probably be good to eschew the use of functions (at least in the beginning). e.g.: using /*:link[@id="XYZ"] or we could strip off the new for the namespace and have /link[@id="XYZ"] .
To keep compatibility I'll replace xquery statements with xpaths (we have started doing this).
6. In the TSQueryResponse example, it looks like far more is returned than I'd expect given the query. Was this just a copy-and-paste instance?
Yes, you're right (SIP->MA(t)). There's a short comment in the request why.
7. The TSAddRequest wraps the individual network elements in an nml:topology element. In the TSQueryResponse response, the individual elements are wrapped in a 'datum'. It'd probably make sense to have the TSAddRequest wrap the individual elements in a 'datum' as well.
Fine for me (I didn't do it this way because I followed the example http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-TopologySer... where there's no datum element)
Yeah, I figured as such. I'd just like to change the semantics to be more similar to other services.
8. What kinds of additional info are you thinking about in 9.2.1.9? Is this stuff that is expected to be retrievable by the client, or is it stuff that the MA is going to strip out?
Additional info in parameters may contain some general info about the reservation and its monitoring (for example: duration) but now I don't have anything specific in my mind.
9. I'm curious what kinds of subjects are envisioned for the SIP to MP messages.
I don't think the MDM release has an MP service (collecting the link status) ready to be used so it will be developed sooner or later. It's difficult now to say how a possible request will look like (not sure I'll be involved in the work on messages for it).
10. In 9.2.2.1, the client does a request for every service containing anything on pionier.net. That could easily become non-scalable. It might make sense to qualify the request a bit (e.g. by including eventTypes of interest):
This request goes to the gLS so only address(es) of hLS(es) of pionier domain will be provided (I don't expect a domain may have many hLS(es)). But of course additional filtering info may be used (depends on what is summarized and sent by the hLS to the gLS).
Well, the result would be any hLS that has any data on the pionier domain. So if there are other domains out there that running PingER, HADES, owamp or bwctl tests against pionier hosts, they'd be returned as well. On the flipside, I'm not sure adding the eventType would help this situation since many of those domains could also have utilization data (which may or may not be applicable to the pionier domain), so a search for utilization data and pionier could grab the same set of domains. I'm wondering if the LS summarization should account for the relationships between the data and the domains. The semantics of the summary query to the gLS would be "give me the hLSes that know about services with utilization data about internet2.edu", as opposed to, "give me the hLSes that know about services with data about internet2.edu, and services with utilization data).
<nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"><summary:subject> <summary:subject> <nml:domain id="pionier.net"/> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata>
11. Also, Jason noticed that the LS response has a datum element that it shouldn't:
<nmwg:message> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"> <summary:subject> <nml:domain id="psu.edu" /> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668"> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="3b010db87b462c778bd017447fcc762c"> <perfsonar:subject xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/"> <psservice:service xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <psservice:serviceName>Lookup Service</psservice:serviceName> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint> <psservice:serviceType>hLS</psservice:serviceType> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn State in University Park, PA USA</psservice:serviceDescription> </psservice:service> </perfsonar:subject> </nmwg:metadata> </nmwg:data> </nmwg:message>
I'll update it.
thanks, Roman
Cheers, Aaron
Hi Roman/All, On Apr 1, 2011, at 8:25 AM, Roman Łapacz wrote:
Hi Aaron/All,
I'm sending an example of extended lookup information (this is what we discussed yesterday; two new things: eventType and message types in the lookup info):
<?xml version="1.0" encoding="UTF-8"?> <nmwg:message id="msg1" type="LSRegisterRequest" xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/" xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/">
<nmwg:metadata id="meta1"> <perfsonar:subject id="1"> <psservice:service id="mat_1"> <psservice:serviceName>Measurement Archive for intra-domain topology</psservice:serviceName> <psservice:accessPoint>http://host:8080/services/ma_t</psservice:accessPoint> <psservice:serviceType>MA</psservice:serviceType> <psservice:organization>PIONIER</psservice:organization> </psservice:service> </perfsonar:subject> <!-- eventType indicates what kind of data the service supports --> <nmwg:eventType>http://ggf.org/ns/nmwg/topology/201103</nmwg:eventType> <psservice:parameters> <nmwg:parameter name="supportedMessage">TSAddRequest</nmwg:parameter> <nmwg:parameter name="supportedMessage">TSQueryRequest</nmwg:parameter> </psservice:parameters> </nmwg:metadata>
<!-- There is no data element(s) with metadata information because --> <!-- at the beginning (when the service starts) MA(t) does not store any data --> <nmwg:data id="data1" metadataIdRef="meta1"/>
</nmwg:message>
I think the supportedMessage and eventType elements should go into the service element since it is a property of the service itself in the same way that 'name' or 'accessPoint' is. If nothing else, the eventType in a metadata should describe the contents of the data element, and in the above case, the 'data' elements would all be metadata, not topology elements. If anything, I'd rather see the eventType for an LSRegisterRequest be something like "http://ggf.org/ns/nmwg/metadata/1.0" or similar. I'd propose the 'service' element look like: <psservice:service id="mat_1"> <psservice:serviceName>Measurement Archive for intra-domain topology</psservice:serviceName> <psservice:accessPoint>http://host:8080/services/ma_t</psservice:accessPoint> <psservice:serviceType>MA</psservice:serviceType> <psservice:organization>PIONIER</psservice:organization> <psservice:parameters> <psservice:parameter name="supportedMessage">TSAddRequest</psservice:parameter> <psservice:parameter name="supportedMessage">TSQueryRequest</psservice:parameter> <psservice:parameter name="supportedEventType">http://ggf.org/ns/nmwg/topology/201103</psservice:parameter> </psservice:parameters> </psservice:service> The full message would then look like a normal LSRegisterRequest: <?xml version="1.0" encoding="UTF-8"?> <nmwg:message id="msg1" type="LSRegisterRequest" xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/" xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <nmwg:metadata id="meta1"> <perfsonar:subject id="1"> <psservice:service id="mat_1"> <psservice:serviceName>Measurement Archive for intra-domain topology</psservice:serviceName> <psservice:accessPoint>http://host:8080/services/ma_t</psservice:accessPoint> <psservice:serviceType>MA</psservice:serviceType> <psservice:organization>PIONIER</psservice:organization> <psservice:parameters> <psservice:parameter name="supportedMessage">TSAddRequest</psservice:parameter> <psservice:parameter name="supportedMessage">TSQueryRequest</psservice:parameter> <psservice:parameter name="supportedEventType">http://ggf.org/ns/nmwg/topology/201103</psservice:parameter> </psservice:parameters> </psservice:service> </perfsonar:subject> </nmwg:metadata> <!-- There is no data element(s) with metadata information because --> <!-- at the beginning (when the service starts) MA(t) does not store any data --> <nmwg:data id="data1" metadataIdRef="meta1"/> </nmwg:message> Cheers, Aaron
The problem I see is that the MA and the MP may use the same message type (SetupDataRequest) for different actions. In case of the MA it is fetching data. For the MP it is executing a test or collecting data. I don't think we've had a separate message type defined for the MP and if I'm not wrong SetupDataRequest was used (until it is not clarified I prefer to have 'operation' parameter).
Cheers, Roman
W dniu 2011-03-29 17:57, Aaron Brown pisze:
Hey Roman,
Comments inline.
Cheers, Aaron
On Mar 29, 2011, at 8:31 AM, Roman Łapacz wrote:
W dniu 2011-03-28 14:23, Aaron Brown pisze:
On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
Hi,
I'm sending updated document of MDM circuit monitoring.
Aaron, Jason, all, you can find a simplified version of client access algorithm (section 5.3.2) but it still supports that one that you formulated some time ago and we all accepted. It is simplified because some assumptions have been made in the use case with AutoBAHN (see the picture in 5.3.2). Please, take a look at the examples of messages for this algorithm but also for the workflow presented in 5.2.1. Check if massages from and to the MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take a look at the example of massage that is sent from the hLS to the gLS (section 9.2.3).
Some time ago we were talking about capabilities and dropping the service type field in the lookup information. I found it very useful while working on the examples of messages. Although the MA services (MA(at) and MA(t)) store the topology information they can be treated as the pS-PS TS if they support topology storage functionality and register that with the hLS. Take a look at the examples of LSRegisterRequest type (I've used the parameter eventType but probably normal eventType element would be more suitable; but first I'd like to see your comments). So far I've proposed 6 capabilities:
http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA) http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0 (for MA with data aggregated by AutoBAHN) http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP) http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which connects pS with AutoBAHN) http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t) which store topology information) http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0 (for TS or MA (at) which store the topology information aggregated by AutoBAHN)
So the proposal uses eventTypes for operations (e.g. http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this services can be used to store measurements"). My recollection is that the goal was to move away from using eventTypes as operations; they'd just be used to describe functions. To do this, we'd need some way of including the supported messages in the elements registered up by the services. The clients could then look for things like "the services that support MetadataStoreRequest and utilization data". I'm not positive the best way forward on this.
I would not use message types because they also might be misleading like service types. I like the idea of registering the operations that are supported by a service. Changing a bit your example: "the services that support store operation of utilization data".
I guess the open question here is what meaning service type (or supported operation) is meant to convey. Is there an instance where two different services would support the same data types and the same messages, but have different semantics?
As to the parameter-based eventTypes, I think we need to move away from those if, for no other reason, than to ensure that clients can uniformly query the LS without having to worry about what form the eventType will take. Having multiple ways, especially if different software packages do things differently, is more likely to result in clients that can only work with one type or the other.
I've used eventType parameters and I knew this wasn't a good solution but I'd like to see your comments on this. The parameter 'operation' would be more suitable. What do you think?
A few other comments and questions on the document after a first pass.
1. We'll need to think about the specifics of how to generate the segment identifiers when there's a OSCARS/AutoBAHN bridging. When this happens, there are 2 very different identifier schemes, and it's not clear how an agent would go about generating the circuit descriptor since they need to know how other domains are going to generate their segment identifiers.
But the GLIF format and its domain part isn't enough? An agent does not have to know what the last part of the GLIF urn means. (I'll forward this comment to the AutoBAHN team)
In the model, the starting domain creates the circuit descriptor. However, to do that, that domain needs to know how the other domains are going to set the identifiers for their segment descriptors. There are two ways to do that:
1) have a programmatic way that each domain uses to create the segment id using information that is available to all domains 2) have the starting domain contact each of the other domains and request the segment identifiers
I'd prefer #1 if possible since it makes it substantially easier to deploy. However, for #1, there's not a good way, if OSCARS is the starting domain, to figure out how an AutoBAHN domain is going to name its segments since all OSCARS will have is a GRI (not compatible with AutoBAHN's id structure), and the domain names.
2. In the segment and circuit descriptors, there are two different relationship names used: 'connects' and 'serialcompound'. In both cases, the relationship is describing the constituent elements that make up the link (links in the case of the circuit descriptor, and ports in the case of the segment descriptor). Since they're describing the same conceptual relationship, having the same name would be good.
Agree. I was thinking about this while writing descriptor examples in the doc but which one to choose? I would take "connects" but this must be agreed by all of us. So are you fine to take "connects"?
I'd just been using "over", but would be fine with "connects".
3. It looks like the SIP agent is registering the segment ID. I'm curious why it's being registered there.
The SIP contains info about running monitoring sessions and stores it after they are finished to keep the history (detailed info - segment descriptors - can be fetched from the MA(t) or the MA(at)).
4. All of the XQuery requests to the hLS are of the form "what services contain eventType X?". This type of request can be handled using the summary requests which has the side benefit of not embedding more XQuery into protocol.
5. Relatedly, I'd like to specify that XPath queries be the only ones used the TS. This way there's a hope of moving away from the XQuery-based protocols, and their dependency on an XML Database. Since XPath can be converted to SQL requests, that'd be much easier to transition to something else. It'd probably also be good to specify exactly the subset of XPath to be supported so that we don't hit the XQuery issue where you are required to have a full-blown XPath implementation to implement the protocol. To keep the XPath syntax simplified, it'd probably be good to eschew the use of functions (at least in the beginning). e.g.: using /*:link[@id="XYZ"] or we could strip off the new for the namespace and have /link[@id="XYZ"] .
To keep compatibility I'll replace xquery statements with xpaths (we have started doing this).
6. In the TSQueryResponse example, it looks like far more is returned than I'd expect given the query. Was this just a copy-and-paste instance?
Yes, you're right (SIP->MA(t)). There's a short comment in the request why.
7. The TSAddRequest wraps the individual network elements in an nml:topology element. In the TSQueryResponse response, the individual elements are wrapped in a 'datum'. It'd probably make sense to have the TSAddRequest wrap the individual elements in a 'datum' as well.
Fine for me (I didn't do it this way because I followed the example http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-TopologySer... where there's no datum element)
Yeah, I figured as such. I'd just like to change the semantics to be more similar to other services.
8. What kinds of additional info are you thinking about in 9.2.1.9? Is this stuff that is expected to be retrievable by the client, or is it stuff that the MA is going to strip out?
Additional info in parameters may contain some general info about the reservation and its monitoring (for example: duration) but now I don't have anything specific in my mind.
9. I'm curious what kinds of subjects are envisioned for the SIP to MP messages.
I don't think the MDM release has an MP service (collecting the link status) ready to be used so it will be developed sooner or later. It's difficult now to say how a possible request will look like (not sure I'll be involved in the work on messages for it).
10. In 9.2.2.1, the client does a request for every service containing anything on pionier.net. That could easily become non-scalable. It might make sense to qualify the request a bit (e.g. by including eventTypes of interest):
This request goes to the gLS so only address(es) of hLS(es) of pionier domain will be provided (I don't expect a domain may have many hLS(es)). But of course additional filtering info may be used (depends on what is summarized and sent by the hLS to the gLS).
Well, the result would be any hLS that has any data on the pionier domain. So if there are other domains out there that running PingER, HADES, owamp or bwctl tests against pionier hosts, they'd be returned as well. On the flipside, I'm not sure adding the eventType would help this situation since many of those domains could also have utilization data (which may or may not be applicable to the pionier domain), so a search for utilization data and pionier could grab the same set of domains. I'm wondering if the LS summarization should account for the relationships between the data and the domains. The semantics of the summary query to the gLS would be "give me the hLSes that know about services with utilization data about internet2.edu", as opposed to, "give me the hLSes that know about services with data about internet2.edu, and services with utilization data).
<nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"><summary:subject> <summary:subject> <nml:domain id="pionier.net"/> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata>
11. Also, Jason noticed that the LS response has a datum element that it shouldn't:
<nmwg:message> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851"> <summary:subject> <nml:domain id="psu.edu" /> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> </summary:subject> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType> </nmwg:metadata> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668"> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="3b010db87b462c778bd017447fcc762c"> <perfsonar:subject xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/"> <psservice:service xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"> <psservice:serviceName>Lookup Service</psservice:serviceName> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint> <psservice:serviceType>hLS</psservice:serviceType> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn State in University Park, PA USA</psservice:serviceDescription> </psservice:service> </perfsonar:subject> </nmwg:metadata> </nmwg:data> </nmwg:message>
I'll update it.
thanks, Roman
Cheers, Aaron
participants (3)
-
Aaron Brown -
Jason Zurawski -
Roman Łapacz