public comment "Non-deterministic outcome of setVariable evaluations"
Some email discussion before posting to the public comment forum may be helpful for this issue. Right now the rules say that all dfdl:newVariableInstance annotations for a schema component are always executed before any setVariable annotations. This is ok. One can introduce an extra sequence in order to allow a setVariable to occur before a newVariableInstance. (Personally: I would like to get a warning from my DFDL implementation if I write statement annotations at one annotation point that are not in the canonical order described in Errata 3.25) The problem is that the order of execution of multiple setVariable statements at one annotation point, or similarly, of multiple newVariableInstance at one annotation point, is not specified. This is what will get users into trouble. If there are multiple newVariableInstance annotations, then the order is important to understanding when exactly a new instance comes into scope, and whether the prior-scope's value for that variable can be read as part of the default value expressions. Consider this: <dfdl:newVariableInstance ref="save">{ $delim }</dfdl:newVariableInstance> <dfdl:newVariableInstance ref="delim">{ ../hdr/d }</dfdl:newVariableInstance> We need to specify whether $delim in the first annotation is referring to that variable before the new instance is created, or after. I suggest that before is more intuitive, as that is lexical order. Note however, that this means: <dfdl:newVariableInstance ref="prefix">{ fn:concat($prefix, "_", fn:name()) }</dfdl:newVariableInstance> is allowed and is not recursion. The expression is referring to $prefix from the enclosing scope, and then initializing a new instance of $prefix for the scope contained by this schema component. The same logic applies to dfdl:setVariable annotations. The order needs to be controlled by the user. I suggest the fix for this issue is as follows: Errata 3.20, 3.21, and Errata 3.25 must be modified (and the corresponding sections of the spec) For errata 3.20, and errata 3.21 the sentence: "However, the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely." is replaced by: "The order of execution is specified in errata 3.25." Errata 3.25 is modified. Numbered item 3 of both the first list and second list is replaced by: 3. dfdl:setVariable(s) - in lexical order Numbered item 2 of the second list is replaced by: 2. dfdl:newVariableInstance(s) - in lexical order These two paragraphs are added below the numbered lists to clarify what 'lexical order' means. The dfdl:setVariable annotations at any one annotation point of the schema are always executed in lexical order. However, dfdl:setVariable annotations can also be found in different annotation points that are combined into the resolved set of annotations for one schema component. In this case, the order of execution of the dfdl:setVariable statements from any one annotation point remains lexical, but it is unspecified what the order of execution is across dfdl:setVariable statements coming from different annotation points. The dfdl:newVariableInstance annotations at any one annotation point of the schema are always executed in lexical order. However, dfdl:newVariableInstance annotations can also be found in different annotation points that are combined into the resolved set of annotations for one schema component. In this case, the order of execution of the dfdl:newVariableInstance statements from any one annotation point remains lexical, but it is unspecified what the order of execution is across dfdl:newVariableInstance statements coming from different annotation points. ---------------------- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy<http://www.ogf.org/About/abt_policies.php>
I think this leaves us in a strange halfway house where the spec says which order annotations on the same annotation point are executed, but does not say in which order annotations on different annotation points are executed. To my mind, we either have a predictable order or we don't. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Date: 30/09/2013 23:12 Subject: [DFDL-WG] public comment "Non-deterministic outcome of setVariable evaluations" Sent by: dfdl-wg-bounces@ogf.org Some email discussion before posting to the public comment forum may be helpful for this issue. Right now the rules say that all dfdl:newVariableInstance annotations for a schema component are always executed before any setVariable annotations. This is ok. One can introduce an extra sequence in order to allow a setVariable to occur before a newVariableInstance. (Personally: I would like to get a warning from my DFDL implementation if I write statement annotations at one annotation point that are not in the canonical order described in Errata 3.25) The problem is that the order of execution of multiple setVariable statements at one annotation point, or similarly, of multiple newVariableInstance at one annotation point, is not specified. This is what will get users into trouble. If there are multiple newVariableInstance annotations, then the order is important to understanding when exactly a new instance comes into scope, and whether the prior-scope's value for that variable can be read as part of the default value expressions. Consider this: <dfdl:newVariableInstance ref="save">{ $delim }</dfdl:newVariableInstance> <dfdl:newVariableInstance ref="delim">{ ../hdr/d }</dfdl:newVariableInstance> We need to specify whether $delim in the first annotation is referring to that variable before the new instance is created, or after. I suggest that before is more intuitive, as that is lexical order. Note however, that this means: <dfdl:newVariableInstance ref="prefix">{ fn:concat($prefix, "_", fn:name()) }</dfdl:newVariableInstance> is allowed and is not recursion. The expression is referring to $prefix from the enclosing scope, and then initializing a new instance of $prefix for the scope contained by this schema component. The same logic applies to dfdl:setVariable annotations. The order needs to be controlled by the user. I suggest the fix for this issue is as follows: Errata 3.20, 3.21, and Errata 3.25 must be modified (and the corresponding sections of the spec) For errata 3.20, and errata 3.21 the sentence: "However, the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely." is replaced by: "The order of execution is specified in errata 3.25." Errata 3.25 is modified. Numbered item 3 of both the first list and second list is replaced by: 3. dfdl:setVariable(s) - in lexical order Numbered item 2 of the second list is replaced by: 2. dfdl:newVariableInstance(s) - in lexical order These two paragraphs are added below the numbered lists to clarify what 'lexical order' means. The dfdl:setVariable annotations at any one annotation point of the schema are always executed in lexical order. However, dfdl:setVariable annotations can also be found in different annotation points that are combined into the resolved set of annotations for one schema component. In this case, the order of execution of the dfdl:setVariable statements from any one annotation point remains lexical, but it is unspecified what the order of execution is across dfdl:setVariable statements coming from different annotation points. The dfdl:newVariableInstance annotations at any one annotation point of the schema are always executed in lexical order. However, dfdl:newVariableInstance annotations can also be found in different annotation points that are combined into the resolved set of annotations for one schema component. In this case, the order of execution of the dfdl:newVariableInstance statements from any one annotation point remains lexical, but it is unspecified what the order of execution is across dfdl:newVariableInstance statements coming from different annotation points. ---------------------- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Per call on 2013-10-01 revise wording to specify exact order of evaluation when combining newVariableInstance, and setVariable across multiple annotation points that are resolved together. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy<http://www.ogf.org/About/abt_policies.php> On Tue, Oct 1, 2013 at 8:16 AM, Steve Hanson <smh@uk.ibm.com> wrote:
I think this leaves us in a strange halfway house where the spec says which order annotations on the same annotation point are executed, but does not say in which order annotations on different annotation points are executed. To my mind, we either have a predictable order or we don't.
Regards
Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> IBM SWG, Hursley, UK* **smh@uk.ibm.com* <smh@uk.ibm.com> tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Date: 30/09/2013 23:12 Subject: [DFDL-WG] public comment "Non-deterministic outcome of setVariable evaluations" Sent by: dfdl-wg-bounces@ogf.org ------------------------------
Some email discussion before posting to the public comment forum may be helpful for this issue.
Right now the rules say that all dfdl:newVariableInstance annotations for a schema component are always executed before any setVariable annotations. This is ok. One can introduce an extra sequence in order to allow a setVariable to occur before a newVariableInstance.
(Personally: I would like to get a warning from my DFDL implementation if I write statement annotations at one annotation point that are not in the canonical order described in Errata 3.25)
The problem is that the order of execution of multiple setVariable statements at one annotation point, or similarly, of multiple newVariableInstance at one annotation point, is not specified. This is what will get users into trouble.
If there are multiple newVariableInstance annotations, then the order is important to understanding when exactly a new instance comes into scope, and whether the prior-scope's value for that variable can be read as part of the default value expressions. Consider this:
<dfdl:newVariableInstance ref="save">{ $delim }</dfdl:newVariableInstance> <dfdl:newVariableInstance ref="delim">{ ../hdr/d }</dfdl:newVariableInstance>
We need to specify whether $delim in the first annotation is referring to that variable before the new instance is created, or after. I suggest that before is more intuitive, as that is lexical order. Note however, that this means:
<dfdl:newVariableInstance ref="prefix">{ fn:concat($prefix, "_", fn:name()) }</dfdl:newVariableInstance>
is allowed and is not recursion. The expression is referring to $prefix from the enclosing scope, and then initializing a new instance of $prefix for the scope contained by this schema component.
The same logic applies to dfdl:setVariable annotations. The order needs to be controlled by the user.
I suggest the fix for this issue is as follows:
Errata 3.20, 3.21, and Errata 3.25 must be modified (and the corresponding sections of the spec)
For errata 3.20, and errata 3.21 the sentence:
"However, the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely."
is replaced by:
"The order of execution is specified in errata 3.25."
Errata 3.25 is modified.
Numbered item 3 of both the first list and second list is replaced by:
3. dfdl:setVariable(s) - in lexical order
Numbered item 2 of the second list is replaced by:
2. dfdl:newVariableInstance(s) - in lexical order
These two paragraphs are added below the numbered lists to clarify what 'lexical order' means.
The dfdl:setVariable annotations at any one annotation point of the schema are always executed in lexical order. However, dfdl:setVariable annotations can also be found in different annotation points that are combined into the resolved set of annotations for one schema component. In this case, the order of execution of the dfdl:setVariable statements from any one annotation point remains lexical, but it is unspecified what the order of execution is across dfdl:setVariable statements coming from different annotation points.
The dfdl:newVariableInstance annotations at any one annotation point of the schema are always executed in lexical order. However, dfdl:newVariableInstance annotations can also be found in different annotation points that are combined into the resolved set of annotations for one schema component. In this case, the order of execution of the dfdl:newVariableInstance statements from any one annotation point remains lexical, but it is unspecified what the order of execution is across dfdl:newVariableInstance statements coming from different annotation points.
----------------------
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | * www.tresys.com* <http://www.tresys.com/> Please note: Contributions to the DFDL Workgroup's email discussions are subject to the *OGF Intellectual Property Policy*<http://www.ogf.org/About/abt_policies.php> -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (2)
-
Mike Beckerle -
Steve Hanson