Author: zurawski Date: 2010-05-27 13:15:38 -0500 (Thu, 27 May 2010) New Revision: 24 Modified: chaining.tex extension.tex nmc-base.pdf result_codes.tex Log: Completing removing the CAPed BFed RFC 2119 words. Made a couple of more language adjustments as I went through. -jason Modified: chaining.tex =================================================================== --- chaining.tex 2010-05-27 14:31:48 UTC (rev 23) +++ chaining.tex 2010-05-27 18:15:38 UTC (rev 24) @@ -1,11 +1,11 @@ \section{Information Chaining} \label{s:information_chaining} -Since inception a key goal has been extension. The authors of the original schemata realized that not every situation could easily be described though the basic constructs; extending the basic building blocks to complex situations \textbf{MUST} be paramount. Uncharted concepts could be represented with newly created constructs each time a foreign abstraction came to light; but extension and backwards compatibility \textbf{MUST} be favored over quick and easy solutions. Therefore, basic extension mechanisms, known as chaining, are the recognized procedure to extend metadata constructs as well as express other operations on the underlying data. +Since inception, a key goal of the protocols has been extension. The authors of the original schemata realized that not every situation could easily be described though the basic constructs; extending the basic building blocks to complex situations is paramount. Uncharted concepts could be represented with newly created constructs each time a foreign abstraction came to light; but extension and backwards compatibility must be favored over quick and easy solutions. Therefore, basic extension mechanisms, known as chaining, are the recognized procedure to extend metadata constructs as well as express other operations on the underlying data. -This section presents the major uses of chaining; note that individual service implementations \textbf{MAY} choose to strictly or loosely interpret these guidelines for the sake of performance or protection. The protocol itself offers no specific guidance on these issues in favor of simply describing the structural composition of both the input data and the resulting output. +This section presents the major uses of chaining; note that individual service implementations may choose to strictly or loosely interpret these guidelines for the sake of performance or protection. The protocol itself offers no specific guidance on these issues in favor of simply describing the structural composition of both the input data and the resulting output. -Chaining itself has taken on two major forms: \textit{merge chaining} described in Section~\ref{s:information_chaining_merge} and \textit{filter chaining} described in Section~\ref{s:information_chaining_filter}. These two instances will be described first in broad terms that explain the logic and reasoning of why each operation makes sense, and in what context they \textbf{SHOULD} be employed. The specific syntax and transformation steps will be presented in the next section. +Chaining itself has taken on two major forms: \textit{merge chaining} described in Section~\ref{s:information_chaining_merge} and \textit{filter chaining} described in Section~\ref{s:information_chaining_filter}. These two instances will be described first in broad terms that explain the logic and reasoning of why each operation makes sense, and in what context they should be employed. The specific syntax and transformation steps will be presented in the next section. \subsection{Merge Chaining} \label{s:information_chaining_merge} @@ -21,12 +21,12 @@ As stated previously, the schemata itself does not offer any suggestions as to what is a \textit{good merge} vs. a \textit{bad merge}. There are no rules regarding which \textit{types} of data should and should not be merged. There is no guidance on when we should duplicate or replace elements. -We recommend some very simple and succinct guidelines that services may implement for this particular style of merging. There shall always be exceptions to rules, therefore the reader is encouraged to think carefully about what a specific service may need when implementing this recommendation. +We recommend some very simple and succinct guidelines that services may implement for this particular style of merging. There shall be exceptions to rules, therefore the reader is encouraged to think carefully about what a specific service may need when implementing this recommendation. \subsubsection{Mergeable Elements and Recursion} \label{s:information_chaining_merge_elements} -When merging we \textbf{SHOULD} first look at the \textit{top-level} elements; namely subject, eventType, and parameters. When faced with two metadata blocks to be merged, we only wish to combine: +When merging we must first look at the \textit{top-level} elements; namely subject, eventType, and parameters. When faced with two metadata blocks to be merged, we only wish to combine: \begin{itemize} \item \textit{Like} Elements (e.g. sharing the same localname) @@ -34,16 +34,16 @@ \item Elements sharing the same (or ``similar'') eventType \end{itemize} -When this first criteria is met, we \textbf{SHOULD} traverse the chain \textit{downward}, recursively. A clear question to answer is ``How far should we venture into the XML structure looking for similarities or differences''? This question does not have a definite answer such as ``Stop at the grandchild of the current element''. While this \textbf{MAY} be frustrating, domain knowledge \textbf{MAY} help you make a passable decision especially with regards to topology based elements. +When this first criteria is met, we must traverse the chain \textit{downward}, recursively. A clear question to answer is ``How far should we venture into the XML structure looking for similarities or differences''? This question does not have a definite answer such as ``Stop at the grandchild of the current element''. While this may be frustrating, domain knowledge shall help you make a passable decision especially with regards to topology based elements. -\textit{Like} elements that do not share a common namespace \textbf{SHALL} require special rules that \textbf{MAY} differ from service to service. Depending on the level of protection or speed we wish to attain, these rules \textbf{MAY} vary. Service and protocol documentation \textbf{SHALL} fill in details beyond the scope of this work. +\textit{Like} elements that do not share a common namespace will require special rules that may differ from service to service. Depending on the level of protection or speed we wish to attain, these rules may vary. Service and protocol documentation must fill in details beyond the scope of this work. \subsubsection{Duplication, Augmentation, and Replacement} \label{s:information_chaining_merge_dupe} -When are faced with \textit{like} elements that \textbf{MAY NOT} share a common namespace, we \textbf{SHOULD NOT} combine. We \textbf{MAY} try to find the \textit{least significant} namespace and work from there. Additionally we \textbf{MAY} run into items that are \textit{exactly} the same (such as certain \textit{parameters}, or \textit{eventTypes}). In some cases we \textbf{SHOULD} take care to \textit{add} all of these together to make duplicates; other cases \textbf{MAY} dictate total replacement. Specific rule such as these are best left to a service designer. +When are faced with \textit{like} elements that do not share a common namespace, we should not combine. We must try to find the \textit{least significant} namespace and work from there. Additionally we may run into items that are \textit{exactly} the same (such as certain \textit{parameters}, or \textit{eventTypes}). In some cases we should take care to \textit{add} all of these together to make duplicates; other cases may dictate total replacement. Specific rule such as these are best left to a service designer. -As an example of extreme cases, consider taking a very safe approach to the combining of elements (i.e. not merging \textit{like} elements with different namespaces). This approach will ensure that we protect the schema differences but \textbf{MAY} result in many more \textit{wrong} answers when it comes to searching. The converse is a very dangerous approach where we merge items that could be different on the inside. This \textbf{MAY} result in an approach similar to \textit{I know what you meant} and could yield a more robust query mechanism (providing intuitive answers when something may not completely match, rejecting outright things that do not make sense). +As an example of extreme cases, consider taking a very safe approach to the combining of elements (i.e. not merging \textit{like} elements with different namespaces). This approach will ensure that we protect the schema differences but may result in many more \textit{wrong} answers when it comes to searching. The converse is a very dangerous approach where we merge items that could be different on the inside. This may result in an approach similar to \textit{I know what you meant} and could yield a more robust query mechanism (providing intuitive answers when something may not completely match, rejecting outright things that do not make sense). \subsubsection{Merge Chaining Examples} \label{s:information_chaining_merge_examples} @@ -155,7 +155,7 @@ \end{verbatim} \normalsize -For continuity, this example has not attempted to modify the \textit{metadataIdRef} attribute. Implementations \textbf{MAY} choose to do so if they feel the need. Because eventTypes \textbf{MAY} be repeated (either as the \textit{eventType} element or as \textit{parameters}) we \textbf{MUST} take special care when merging them. The next example features multiple eventType merging. This example also features a so called \textit{double chain} where the results of the first chaining operation \textbf{SHALL} feed into the process for the second. This is a common occurrence, and \textbf{SHOULD} be supported in services. +For continuity, this example has not attempted to modify the \textit{metadataIdRef} attribute. Implementations may choose to do so if they feel the need. Because eventTypes may be repeated (either as the \textit{eventType} element or as \textit{parameters}) we must take special care when merging them. The next example features multiple eventType merging. This example also features a so called \textit{double chain} where the results of the first chaining operation must feed into the process for the second. This is a common occurrence, and should be supported in services. \tiny \begin{verbatim} @@ -185,7 +185,7 @@ \end{verbatim} \normalsize -The resulting output and cartoon are pictured below. We did take two major issues into consideration: multiple \textit{parameters} and \textit{eventType} elements that did conflict, and the double chaining. Services that do not support multiple eventTypes (or simply wish to not implement a naive form of chaining) \textbf{SHOULD NOT} worry about special cases such as Figure~\ref{fig:chain2}. +The resulting output and cartoon are pictured below. We did take two major issues into consideration: multiple \textit{parameters} and \textit{eventType} elements that did conflict, and the double chaining. Services that do not support multiple eventTypes (or simply wish to not implement a naive form of chaining) should not worry about special cases such as Figure~\ref{fig:chain2}. \begin{figure}[ht] \centering @@ -245,7 +245,7 @@ \end{verbatim} \normalsize -Services \textbf{MAY} treat particular elements (such as eventTypes and parameters with certain \textit{name} attributes) in a special way. The service is careful not to overwrite or lose any information and will only \textit{add} these items together. This is not the case for any element though, consider the following example. +Services may treat particular elements (such as eventTypes and parameters with certain \textit{name} attributes) in a special way. The service is careful not to overwrite or lose any information and will only \textit{add} these items together. This is not the case for any element though, consider the following example. \tiny \begin{verbatim} @@ -312,7 +312,7 @@ \end{verbatim} \normalsize -This example shows that it is very easy to introduce semantic errors when designing a chain instance. It also shows that the service \textbf{MAY NOT} be interested in protecting a poorly designed chain from being accepted. It is possible to build in different rules instead of \textit{last seen value} such as \textit{first seen}, \textit{original}, or other combinations. It is imperative that services describe their own implementations of chaining, particularly when interoperability becomes an issue. +This example shows that it is very easy to introduce semantic errors when designing a chain instance. It also shows that the service may not be interested in protecting a poorly designed chain from being accepted. It is possible to build in different rules instead of \textit{last seen value} such as \textit{first seen}, \textit{original}, or other combinations. It is imperative that services describe their own implementations of chaining, particularly when interoperability becomes an issue. A final example comes when we deal with items with the same \textit{localname}, but perhaps a different \textit{namespace}. There are several approaches that can be taken to dealing with this type of situation. The SNMP example follows a safe approach of simply adding all of the elements in question and not attempting to internally merge at all. This causes \textit{unreadable} metadata in many cases, but does not permit \textit{data pollution}. @@ -512,7 +512,7 @@ \end{verbatim} \normalsize -Clearly the two eventTypes (for utilization and errors) \textbf{MAY NOT} appear in the same metadata description, but again the service can try to help out a bit. eventType descriptions are interpreted as \textit{or} operations when performing a query. Therefore even if our chain was constructed poorly, our final results will be rather robust (perhaps a bit more robust than needed). The service designers will no doubt settle on an approach that fits well for the data they are exposing. +Clearly the two eventTypes (for utilization and errors) may not appear in the same metadata description, but again the service can try to help out a bit. eventType descriptions are interpreted as \textit{or} operations when performing a query. Therefore even if our chain was constructed poorly, our final results will be rather robust (perhaps a bit more robust than needed). The service designers will no doubt settle on an approach that fits well for the data they are exposing. \subsection{Filter Chaining} \label{s:information_chaining_filter} @@ -528,7 +528,7 @@ \label{fig:ven} \end{figure} -It is important to note that even though we are manipulating the data through this form of chaining, we \textbf{SHOULD} not be harming it, or the related metadata elements. Chaining in general is a non-destructive operation, although it is very possible that when implemented poorly response data corruption \textbf{MAY} occur. +It is important to note that even though we are manipulating the data through this form of chaining, we should not be harming it, or the related metadata elements. Chaining in general is a non-destructive operation, although it is very possible that when implemented poorly response data corruption may occur. Filter operations themselves can vary from time range selection to aggregations such as performing a cumulative distribution function (CDF). Describing all possible operators is well beyond the scope of this work. Current experience has named most statistical and database operations as candidates for filtering, although new uses being devised. @@ -578,7 +578,7 @@ \end{verbatim} \normalsize -The reference is placed in the \textit{subject} element in this case, as in merge chaining this is a signal to the service that filter chaining \textbf{SHALL} be required. This indicates that the \textit{input} is the data pointed to by the first metadata and the \textit{output} will be a subset of this. For the sake of these examples we \textbf{SHALL} be dealing with the \textit{select} namespace as our filter of choice due to an abundance of examples and its common goal of filtering based on time. Other filter examples \textbf{SHOULD} work in the same manner. +The reference is placed in the \textit{subject} element in this case, as in merge chaining this is a signal to the service that filter chaining may be required. This indicates that the \textit{input} is the data pointed to by the first metadata and the \textit{output} will be a subset of this. For the sake of these examples we will be dealing with the \textit{select} namespace as our filter of choice due to an abundance of examples and its common goal of filtering based on time. Other filter examples should work in the same manner. Because the operations of a filter chain are essentially \textit{internal} we do not present what resultant XML should look like. Currently services ignore many of the steps that may go into reforming the XML for response messages in favor of simply returning the \textit{backend} representation of metadata. While quick and easy, this does lead to information loss (specifically when dealing with the various ways to implement merge chaining). Client applications may have no reason to see the original filter information, and therefore are built not to need it. Modified: extension.tex =================================================================== --- extension.tex 2010-05-27 14:31:48 UTC (rev 23) +++ extension.tex 2010-05-27 18:15:38 UTC (rev 24) @@ -1,7 +1,7 @@ \section{Extension} \label{s:extension} -This document \textbf{MUST} become the basis for all extension protocols. As a demonstration we include a protocol that \textbf{SHOULD} be implemented by all measurement services: the \textit{Echo Protocol}. This protocol will incorporate the preceding work to eliminate duplication as much as possible, only specifying parts that are necessary for clarification. Each protocol extension may be treated as a separate work, and will include the necessary schema, analysis, and example sections. +This section must become the basis for all extension protocols. As a demonstration we include a protocol that must be implemented by all measurement services: the \textit{Echo Protocol}. This protocol will incorporate the preceding work to eliminate duplication as much as possible, only specifying parts that are necessary for clarification. Each protocol extension may be treated as a separate work, and will include the necessary schema, analysis, and example sections. \subsection{Echo Protocol} \label{s:echo_protocol} @@ -17,9 +17,9 @@ \subsubsection{Architecture} \label{s:echo_protocol_architecture} -To ensure availability, each service \textbf{MUST} be able to respond to simple queries regarding status. Services that fail to answer a direct question \textbf{MAY} be experiencing difficulty, and therefore \textbf{MAY NOT} be able to complete interaction with interested parties. Client applications, services, or external monitoring tools \textbf{MAY} use this simple method to quickly come to conclusions regarding framework availability. +To ensure availability, each service must be able to respond to simple queries regarding status. Services that fail to answer a direct question may be experiencing difficulty, and therefore may not be able to complete interaction with interested parties. Client applications, services, or external monitoring tools should use this simple method to quickly come to conclusions regarding framework availability. -All services \textbf{MUST} contain the ability to respond to the most basic of \textit{Echo Protocol} messages as described by this document. The minimum requirement of an \textit{Echo Protocol} exchange is simply responding to a properly encoded request. \textit{Echo Protocol} extensions \textbf{MAY} be built from this general protocol to elicit additional functionality on a service by service basis to do tasks such as test the capabilities of the service, receive statistics, or monitor erroneous behavior. The assignment of these other tasks within an \textbf{EchoRequest} message is valid provided that the basic structure is not compromised. +All services must contain the ability to respond to the most basic of \textit{Echo Protocol} messages as described by this document. The minimum requirement of an \textit{Echo Protocol} exchange is simply responding to a properly encoded request. \textit{Echo Protocol} extensions may be built from this general protocol to elicit additional functionality on a service by service basis to do tasks such as test the capabilities of the service, receive statistics, or monitor erroneous behavior. The assignment of these other tasks within an \textbf{EchoRequest} message is valid provided that the basic structure is not compromised. \subsubsection{Request Message} \label{s:echo_protocol_request} @@ -191,7 +191,7 @@ \end{verbatim} \normalsize -The final example shows two incorrect items: the message \textit{type} and \textit{eventType} are both wrong. This \textbf{MUST} be rejected by a service. +The final example shows two incorrect items: the message \textit{type} and \textit{eventType} are both wrong. This must be rejected by a service. \tiny \begin{verbatim} @@ -460,17 +460,17 @@ \\ \end{tabular} -Note that it is possible to add more specific error conditions as the functionality of this protocol increases. For example, if we are testing database connectivity it \textbf{MAY} make sense to add an \textit{echo.db.} range of status. This is left as an exercise for extensions to this protocol. +Note that it is possible to add more specific error conditions as the functionality of this protocol increases. For example, if we are testing database connectivity it may make sense to add an \textit{echo.db.} range of status. This is left as an exercise for extensions to this protocol. \subsubsection{Protocol Extension} \label{s:echo_protocol_extension} -There are two avenues for extension within the \textit{Echo Protocol} as described in this document. It is possible to manipulate the values contained within the \textit{eventType} to advance functionality, or through \textit{schema modification} it is possible to add additional elements capable of handling a wider range of actions. Extensions that modify the schema for a given service \textbf{SHOULD NOT} change the themes presented in this protocol specification. It is imperative that all services respect the basic functionality in their quest to add new features. +There are two avenues for extension within the \textit{Echo Protocol} as described in this document. It is possible to manipulate the values contained within the \textit{eventType} to advance functionality, or through \textit{schema modification} it is possible to add additional elements capable of handling a wider range of actions. Extensions that modify the schema for a given service must not change the themes presented in this protocol specification. It is imperative that all services respect the basic functionality in their quest to add new features. \paragraph{eventType Extension} \label{s:echo_protocol_extension_eventtype} -The current accepted eventType for the \textit{Echo Protocol}'s \textbf{EchoRequest} message is \\ \textbf{http://schemas.perfsonar.net/tools/admin/echo/2.0}. This \textit{action} \textbf{MUST} be accepted by all services. By adding additional eventTypes with the same format it is possible to extract additional information via a service. +The current accepted eventType for the \textit{Echo Protocol}'s \textbf{EchoRequest} message is \\ \textbf{http://schemas.perfsonar.net/tools/admin/echo/2.0}. This \textit{action} must be accepted by all services. By adding additional eventTypes with the same format it is possible to extract additional information via a service. Consider simple service X. The designer of this service wishes to create a special behaviour for specific eventTypes. The following new eventTypes are added to her service code (and to her implementation of the schema): @@ -486,7 +486,7 @@ \paragraph{Other Extensions} \label{s:echo_protocol_extension_other} -Similar to the above approach, it is possible to extend the schema by adding additional elements to increase functionality. Individuals pursuing this route should be comfortable with schema design in general and the layout of the \textbf{NM-WG} and \textbf{NMC-WG} schema descriptions specifically. +Similar to the above approach, it is possible to extend the schema by adding additional elements to increase functionality. Individuals pursuing this route must be comfortable with schema design in general and the layout of the \textbf{NM-WG} and \textbf{NMC-WG} schema descriptions specifically. A simple extension involves allowing the commonly used parameters structure to reside in the \textbf{Message} of the \textbf{EchoRequest} message. This modification is presented below. @@ -525,5 +525,5 @@ While this method does require some additional schema modification, the result produced is the same as described in Section~\ref{s:echo_protocol_extension_eventtype}. An important consideration is the inclusion of \textit{parameters} in an \textbf{EchoRequest}. -It \textbf{SHOULD} be noted that the extension methods proposed here preserve the underlying base protocol completely. Existing services that provide strict validation \textbf{MAY} reject messages that do not fit this standard explicitly, so be sure to design client applications appropriately. +It must be noted that the extension methods proposed here preserve the underlying base protocol completely. Existing services that provide strict validation may reject messages that do not fit this standard explicitly, so be sure to design client applications appropriately. Modified: nmc-base.pdf =================================================================== (Binary files differ) Modified: result_codes.tex =================================================================== --- result_codes.tex 2010-05-27 14:31:48 UTC (rev 23) +++ result_codes.tex 2010-05-27 18:15:38 UTC (rev 24) @@ -11,7 +11,7 @@ -There are currently two hierarchical systems that \textbf{MAY} be used to return status information about services. Each of these approaches takes into account the facts that there are many diverse services as well as different status messages that \textbf{MAY} be returned. The later system also allows for version control and backwards compatibility that the original attempt did not consider. Either system will accomplish the goal of providing a scalable and readable result code tree. It is recommended that the latter approach be adopted as it contains superior features. +There are currently two hierarchical systems that may be used to return status information about services. Each of these approaches takes into account the facts that there are many diverse services as well as different status messages that should be returned. The later system also allows for version control and backwards compatibility that the original attempt did not consider. Either system will accomplish the goal of providing a scalable and readable result code tree. It is recommended that the latter approach be adopted as it contains superior features. The original idea for result codes comes from the perfSONAR\cite{perfsonar} framework and is explained in \cite{psresult}. This approach relies on a static tree of status information that is branched first by general features (i.e. success, error) and later by more specific characteristics such as service and error type. This hierarchy is missing the ability to offer versions of different codes that could become forward or backwards compatible (seen now in the \textit{eventType} system that takes the form of \textit{URI} strings). @@ -31,7 +31,7 @@ \\ \end{tabular} -The most recent description of this status paradigm follows the URI syntax already in use by other eventTypes and does allow version numbers to be assigned to the various messages. This \textbf{SHALL} allow services to pick and choose which versions they may support more readily. +The most recent description of this status paradigm follows the URI syntax already in use by other eventTypes and does allow version numbers to be assigned to the various messages. This shall allow services to pick and choose which versions they may support more readily. \begin{tabular}{lllll} \\