Action 186: Re: DFDL Statement Evaluation Timing (Assert, Discriminator, SetVariable, NewVariableInstance)
I had the action to create the final errata wording for this item (Action 186). Last action issue was to change wording to insure it was all dealing with combined annotations uniformly, and change "annotated construct" to "schema component". I've done that, but also reorganized this as errata on specific sections of the spec consistent with the rest of our errata, and turned the 'ordering' section into numbered lists for clarity. ------------------------------------------------------- * Errata section 6.2. Clarification:* At any single annotation point of the schema, there can be only one format annotation (dfdl:format, dfdl:element, dfdl:sequence, dfdl:choice, dfdl:group, dfdl:simpleType). *Errata section 3. Glossary entries* *Glossary*: Annotation point: A location within a DFDL schema where DFDL annotation elements are allowed to appear. Note that annotation point is defined in section 6.2, but it belongs also in the glossary. * Glossary*: DFDL Statement annotations, or just DFDL Statements, are the annotation elements dfdl:assert, dfdl:discriminator, dfdl:setVariable, and dfdl:newVariableInstance. *Glossary*: DFDL Defining Annotations are the annotation elements dfdl:defineFormat, dfdl:defineVariable, and dfdl:defineEscapeScheme *Glossary*: DFDL Format Annotations are the annotation elements dfdl:format, dfdl:element, dfdl:simpleType, dfdl:group, dfdl:sequence, and dfdl:choice. *Glossary*: Remove existing definition for "Format Annotations" *Glossary*: Physical layer - revised to: "Physical Layer - A DFDL Schema adds DFDL annotations onto an XSDL language schema. The annotations describe the physical representation or physical layer of the data." * Glossary*: *Combined annotations*: When DFDL annotations appear on a group reference and the sequence or choice of the referenced global group, or appear among an element reference, an element declaration, and its type definition, then they are combined together and the resulting set of annotations is referred to as the *combined annotations* for the schema component.* Errata Section 7.3.1: *Remove "DFDL asserts can be placed on components within a DFDL model." as this is made more specifically subsequently. Replace "More than one dfdl:assert may be used at an annotation point. The dfdl:asserts will be evaluated in the order defined in the schema." with "If the combined annotations for a schema component contain multiple dfdl:assert statements, then those with testKind='pattern' are executed before those with testKind='expression' (the default). However, within each group the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely. See Section REF: *Evaluation Order for Statement Annotations.*" Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. * ** Errata Section 7.3.1, and 7.4.1: Assert and Discriminator Placement* Correct description to be consistent with: dfdl:assert and dfdl:discriminator can be placed as annotations on sequence, choice, group references, local and global element declarations, element references, and simple type definitions. *Errata Section 7.4.1:* Replace: "Any one annotation point can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." with: "The combined annotations for a schema component can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. *Errata Section 7.8 newVariableInstance* dfdl:newVariableInstance can be placed as an annotation on sequence, choice, and group references. Replace: "It is a schema definition error to have more than one newVariableInstances for the same variable at any given point in the document." with "The combined annotations for any schema component can contain multiple dfdl:newVariableInstance annotations, but they must each refer to a different variable. It is a schema definition error otherwise." *Errata Section 7.9: setVariable * Correct language to be consistent with: "dfdl:setVariable may be placed as an annotation on sequence, choice, group references, local and global element declarations for elements of simple type, element references to elements of simple type, and simple type definitions. " Replace "It is a schema definition error to have more than one dfdl:setVariable for the same variable at any given point in the document." with "The combined annotations for any schema component can contain multiple dfdl:setVariable annotations, but they must each refer to a different variable. It is a schema definition error otherwise." * Errata: New Section 7.10: Evaluation Order for Statement Annotations *Of the combined annotations for a schema component, some are statement annotations and the order of their evaluation relative to the actual processing of the schema component itself (parsing or unparsing per its format annotations) is as given in the ordered lists below. Implementations are free to optimize by recognizing and executing discriminators or assertions with testKind='expression' earlier so long as the resulting behavior is consistent with what results from the this description: For elements: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. Processing of the element schema component itself (as per format annotations) 4. dfdl:setVariable(s) - See note below. 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) For sequences and choices: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. dfdl:newVariableInstance(s) 4. dfdl:setVariable(s) 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3 or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) 7. Processing of the sequence/choice component itself (as per format annotations) * Note on Discriminators with testKind='expression' *When parsing, an attempt to evaluate a discriminator must be made even if preceding statements or the parse of the schema component ended in a parse error. This is because a discriminator's expression could evaluate to true thereby resolving a point of uncertainty even if the complete parsing of the construct ultimately caused a parse error. Such discriminator evaluation has access to the DFDL Infoset of the attempted parse as it existed immediately before detecting the parse failure. Attempts to reference parts of the DFDL Infoset that do not exist are parse errors. It is also a parse error to read any variables defined by any dfdl:newVariableInstance statement evaluations that failed, or to read any variables set by any dfdl:setVariable statements that failed. * Note on the dfdl:setVariable Statement for Elements* The combined dfdl:setVariable statements for an element schema com ponent (combined from an element reference, element declaration, and base and derived simpleType definitions) are executed after the parsing of the element. This implies that these variables are set after the evaluation of expressions corresponding to any computed DFDL format properties for the same schema component, and so the variables may not be referenced from expressions that compute DFDL properties. That is, if an expression is used to provide the value of a format property (such as dfdl:terminator, or dfdl:byteOrder), the evaluation of that format property expression occurs before any dfdl:setVariable annotation from the combined annotations for that schema component are executed; hence, the expression providing the value of the format property may not reference the variable. Schema authors can insert sequences to provide more precise control over when variables are set.
Mike, I think we agreed on 'resolved set of annotations' instead of 'combined annotations' ? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: dfdl-wg@ogf.org, Date: 19/11/2012 17:16 Subject: [DFDL-WG] Action 186: Re: DFDL Statement Evaluation Timing (Assert, Discriminator, SetVariable, NewVariableInstance) Sent by: dfdl-wg-bounces@ogf.org I had the action to create the final errata wording for this item (Action 186). Last action issue was to change wording to insure it was all dealing with combined annotations uniformly, and change "annotated construct" to "schema component". I've done that, but also reorganized this as errata on specific sections of the spec consistent with the rest of our errata, and turned the 'ordering' section into numbered lists for clarity. ------------------------------------------------------- Errata section 6.2. Clarification: At any single annotation point of the schema, there can be only one format annotation (dfdl:format, dfdl:element, dfdl:sequence, dfdl:choice, dfdl:group, dfdl:simpleType). Errata section 3. Glossary entries Glossary: Annotation point: A location within a DFDL schema where DFDL annotation elements are allowed to appear. Note that annotation point is defined in section 6.2, but it belongs also in the glossary. Glossary: DFDL Statement annotations, or just DFDL Statements, are the annotation elements dfdl:assert, dfdl:discriminator, dfdl:setVariable, and dfdl:newVariableInstance. Glossary: DFDL Defining Annotations are the annotation elements dfdl:defineFormat, dfdl:defineVariable, and dfdl:defineEscapeScheme Glossary: DFDL Format Annotations are the annotation elements dfdl:format, dfdl:element, dfdl:simpleType, dfdl:group, dfdl:sequence, and dfdl:choice. Glossary: Remove existing definition for "Format Annotations" Glossary: Physical layer - revised to: "Physical Layer - A DFDL Schema adds DFDL annotations onto an XSDL language schema. The annotations describe the physical representation or physical layer of the data." Glossary: Combined annotations: When DFDL annotations appear on a group reference and the sequence or choice of the referenced global group, or appear among an element reference, an element declaration, and its type definition, then they are combined together and the resulting set of annotations is referred to as the combined annotations for the schema component. Errata Section 7.3.1: Remove "DFDL asserts can be placed on components within a DFDL model." as this is made more specifically subsequently. Replace "More than one dfdl:assert may be used at an annotation point. The dfdl:asserts will be evaluated in the order defined in the schema." with "If the combined annotations for a schema component contain multiple dfdl:assert statements, then those with testKind='pattern' are executed before those with testKind='expression' (the default). However, within each group the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely. See Section REF: Evaluation Order for Statement Annotations." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. Errata Section 7.3.1, and 7.4.1: Assert and Discriminator Placement Correct description to be consistent with: dfdl:assert and dfdl:discriminator can be placed as annotations on sequence, choice, group references, local and global element declarations, element references, and simple type definitions. Errata Section 7.4.1: Replace: "Any one annotation point can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." with: "The combined annotations for a schema component can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. Errata Section 7.8 newVariableInstance dfdl:newVariableInstance can be placed as an annotation on sequence, choice, and group references. Replace: "It is a schema definition error to have more than one newVariableInstances for the same variable at any given point in the document." with "The combined annotations for any schema component can contain multiple dfdl:newVariableInstance annotations, but they must each refer to a different variable. It is a schema definition error otherwise." Errata Section 7.9: setVariable Correct language to be consistent with: "dfdl:setVariable may be placed as an annotation on sequence, choice, group references, local and global element declarations for elements of simple type, element references to elements of simple type, and simple type definitions. " Replace "It is a schema definition error to have more than one dfdl:setVariable for the same variable at any given point in the document." with "The combined annotations for any schema component can contain multiple dfdl:setVariable annotations, but they must each refer to a different variable. It is a schema definition error otherwise." Errata: New Section 7.10: Evaluation Order for Statement Annotations Of the combined annotations for a schema component, some are statement annotations and the order of their evaluation relative to the actual processing of the schema component itself (parsing or unparsing per its format annotations) is as given in the ordered lists below. Implementations are free to optimize by recognizing and executing discriminators or assertions with testKind='expression' earlier so long as the resulting behavior is consistent with what results from the this description: For elements: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. Processing of the element schema component itself (as per format annotations) 4. dfdl:setVariable(s) - See note below. 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) For sequences and choices: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. dfdl:newVariableInstance(s) 4. dfdl:setVariable(s) 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3 or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) 7. Processing of the sequence/choice component itself (as per format annotations) Note on Discriminators with testKind='expression' When parsing, an attempt to evaluate a discriminator must be made even if preceding statements or the parse of the schema component ended in a parse error. This is because a discriminator's expression could evaluate to true thereby resolving a point of uncertainty even if the complete parsing of the construct ultimately caused a parse error. Such discriminator evaluation has access to the DFDL Infoset of the attempted parse as it existed immediately before detecting the parse failure. Attempts to reference parts of the DFDL Infoset that do not exist are parse errors. It is also a parse error to read any variables defined by any dfdl:newVariableInstance statement evaluations that failed, or to read any variables set by any dfdl:setVariable statements that failed. Note on the dfdl:setVariable Statement for Elements The combined dfdl:setVariable statements for an element schema com ponent (combined from an element reference, element declaration, and base and derived simpleType definitions) are executed after the parsing of the element. This implies that these variables are set after the evaluation of expressions corresponding to any computed DFDL format properties for the same schema component, and so the variables may not be referenced from expressions that compute DFDL properties. That is, if an expression is used to provide the value of a format property (such as dfdl:terminator, or dfdl:byteOrder), the evaluation of that format property expression occurs before any dfdl:setVariable annotation from the combined annotations for that schema component are executed; hence, the expression providing the value of the format property may not reference the variable. Schema authors can insert sequences to provide more precise control over when variables are set. -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Easily fixed. Global replace done. On Mon, Nov 19, 2012 at 12:36 PM, Steve Hanson <smh@uk.ibm.com> wrote:
Mike, I think we agreed on 'resolved set of annotations' instead of 'combined annotations' ?
------------------------------
I had the action to create the final errata wording for this item (Action 186).
Last action issue was to change wording to insure it was all dealing with combined annotations uniformly, and change "annotated construct" to "schema component".
I've done that, but also reorganized this as errata on specific sections of the spec consistent with the rest of our errata, and turned the 'ordering' section into numbered lists for clarity.
-------------------------------------------------------*
Errata section 6.2. Clarification:* At any single annotation point of the schema, there can be only one format annotation (dfdl:format, dfdl:element, dfdl:sequence, dfdl:choice, dfdl:group, dfdl:simpleType). * Errata section 3. Glossary entries* * Glossary*: Annotation point: A location within a DFDL schema where DFDL annotation elements are allowed to appear.
Note that annotation point is defined in section 6.2, but it belongs also in the glossary.*
Glossary*: DFDL Statement annotations, or just DFDL Statements, are the annotation elements dfdl:assert, dfdl:discriminator, dfdl:setVariable, and dfdl:newVariableInstance. * Glossary*: DFDL Defining Annotations are the annotation elements dfdl:defineFormat, dfdl:defineVariable, and dfdl:defineEscapeScheme * Glossary*: DFDL Format Annotations are the annotation elements dfdl:format, dfdl:element, dfdl:simpleType, dfdl:group, dfdl:sequence, and dfdl:choice. * Glossary*: Remove existing definition for "Format Annotations" * Glossary*: Physical layer - revised to: "Physical Layer - A DFDL Schema adds DFDL annotations onto an XSDL language schema. The annotations describe the physical representation or physical layer of the data."*
Glossary*: *Resolved set of annotations*: When DFDL annotations appear on a group reference and the sequence or choice of the referenced global group, or appear among an element reference, an element declaration, and its type definition, then they are combined together and the resulting set of annotations is referred to as the *resolved set of annotations* for the schema component.*
Errata Section 7.3.1: * Remove "DFDL asserts can be placed on components within a DFDL model." as this is made more specifically subsequently.
Replace "More than one dfdl:assert may be used at an annotation point. The dfdl:asserts will be evaluated in the order defined in the schema." with "If the resolved set of annotations for a schema component contain multiple dfdl:assert statements, then those with testKind='pattern' are executed before those with testKind='expression' (the default). However, within each group the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely. See Section REF: *Evaluation Order for Statement Annotations.*"
Property testPattern: Add this to the description:
Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern.
It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. *
Errata Section 7.3.1, and 7.4.1: Assert and Discriminator Placement*
Correct description to be consistent with: dfdl:assert and dfdl:discriminator can be placed as annotations on sequence, choice, group references, local and global element declarations, element references, and simple type definitions. * Errata Section 7.4.1:*
Replace: "Any one annotation point can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." with:
"The resolved set of annotations for a schema component can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise."
Property testPattern: Add this to the description:
Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern.
It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. * Errata Section 7.8 newVariableInstance*
dfdl:newVariableInstance can be placed as an annotation on sequence, choice, and group references.
Replace: "It is a schema definition error to have more than one newVariableInstances for the same variable at any given point in the document." with "The resolved set of annotations for any schema component can contain multiple dfdl:newVariableInstance annotations, but they must each refer to a different variable. It is a schema definition error otherwise." * Errata Section 7.9: setVariable *
Correct language to be consistent with: "dfdl:setVariable may be placed as an annotation on sequence, choice, group references, local and global element declarations for elements of simple type, element references to elements of simple type, and simple type definitions. "
Replace "It is a schema definition error to have more than one dfdl:setVariable for the same variable at any given point in the document." with "The resolved set of annotations for any schema component can contain multiple dfdl:setVariable annotations, but they must each refer to a different variable. It is a schema definition error otherwise."*
Errata: New Section 7.10: Evaluation Order for Statement Annotations * Of the resolved set of annotations for a schema component, some are statement annotations and the order of their evaluation relative to the actual processing of the schema component itself (parsing or unparsing per its format annotations) is as given in the ordered lists below.
Implementations are free to optimize by recognizing and executing discriminators or assertions with testKind='expression' earlier so long as the resulting behavior is consistent with what results from the this description:
For elements: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. Processing of the element schema component itself (as per format annotations) 4. dfdl:setVariable(s) - See note below. 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) For sequences and choices: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. dfdl:newVariableInstance(s) 4. dfdl:setVariable(s) 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3 or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) 7. Processing of the sequence/choice component itself (as per format annotations) *Note on Discriminators with testKind='expression' * When parsing, an attempt to evaluate a discriminator must be made even if preceding statements or the parse of the schema component ended in a parse error.
This is because a discriminator's expression could evaluate to true thereby resolving a point of uncertainty even if the complete parsing of the construct ultimately caused a parse error.
Such discriminator evaluation has access to the DFDL Infoset of the attempted parse as it existed immediately before detecting the parse failure. Attempts to reference parts of the DFDL Infoset that do not exist are parse errors.
It is also a parse error to read any variables defined by any dfdl:newVariableInstance statement evaluations that failed, or to read any variables set by any dfdl:setVariable statements that failed. * Note on the dfdl:setVariable Statement for Elements*
Of the resolved set of annotations, the dfdl:setVariable statements for an element schema component (combined from an element reference, element declaration, and base and derived simpleType definitions) are executed after the parsing of the element.
This implies that these variables are set after the evaluation of expressions corresponding to any computed DFDL format properties for the same schema component, and so the variables may not be referenced from expressions that compute DFDL properties.
That is, if an expression is used to provide the value of a format property (such as dfdl:terminator, or dfdl:byteOrder), the evaluation of that format property expression occurs before any dfdl:setVariable annotation from the resolved set of annotations for that schema component are executed; hence, the expression providing the value of the format property may not reference the variable. Schema authors can insert sequences to provide more precise control over when variables are set.
Mike, discriminators and asserts with testKind 'expression' are always evaluated after the component's format annotation, the timing does not depend on the component kind. See your original write up. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: dfdl-wg@ogf.org Date: 19/11/2012 17:43 Subject: Re: [DFDL-WG] Action 186: Re: DFDL Statement Evaluation Timing (Assert, Discriminator, SetVariable, NewVariableInstance) Easily fixed. Global replace done. On Mon, Nov 19, 2012 at 12:36 PM, Steve Hanson <smh@uk.ibm.com> wrote: Mike, I think we agreed on 'resolved set of annotations' instead of 'combined annotations' ? I had the action to create the final errata wording for this item (Action 186). Last action issue was to change wording to insure it was all dealing with combined annotations uniformly, and change "annotated construct" to "schema component". I've done that, but also reorganized this as errata on specific sections of the spec consistent with the rest of our errata, and turned the 'ordering' section into numbered lists for clarity. ------------------------------------------------------- Errata section 6.2. Clarification: At any single annotation point of the schema, there can be only one format annotation (dfdl:format, dfdl:element, dfdl:sequence, dfdl:choice, dfdl:group, dfdl:simpleType). Errata section 3. Glossary entries Glossary: Annotation point: A location within a DFDL schema where DFDL annotation elements are allowed to appear. Note that annotation point is defined in section 6.2, but it belongs also in the glossary. Glossary: DFDL Statement annotations, or just DFDL Statements, are the annotation elements dfdl:assert, dfdl:discriminator, dfdl:setVariable, and dfdl:newVariableInstance. Glossary: DFDL Defining Annotations are the annotation elements dfdl:defineFormat, dfdl:defineVariable, and dfdl:defineEscapeScheme Glossary: DFDL Format Annotations are the annotation elements dfdl:format, dfdl:element, dfdl:simpleType, dfdl:group, dfdl:sequence, and dfdl:choice. Glossary: Remove existing definition for "Format Annotations" Glossary: Physical layer - revised to: "Physical Layer - A DFDL Schema adds DFDL annotations onto an XSDL language schema. The annotations describe the physical representation or physical layer of the data." Glossary: Resolved set of annotations: When DFDL annotations appear on a group reference and the sequence or choice of the referenced global group, or appear among an element reference, an element declaration, and its type definition, then they are combined together and the resulting set of annotations is referred to as the resolved set of annotations for the schema component. Errata Section 7.3.1: Remove "DFDL asserts can be placed on components within a DFDL model." as this is made more specifically subsequently. Replace "More than one dfdl:assert may be used at an annotation point. The dfdl:asserts will be evaluated in the order defined in the schema." with "If the resolved set of annotations for a schema component contain multiple dfdl:assert statements, then those with testKind='pattern' are executed before those with testKind='expression' (the default). However, within each group the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely. See Section REF: Evaluation Order for Statement Annotations." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. Errata Section 7.3.1, and 7.4.1: Assert and Discriminator Placement Correct description to be consistent with: dfdl:assert and dfdl:discriminator can be placed as annotations on sequence, choice, group references, local and global element declarations, element references, and simple type definitions. Errata Section 7.4.1: Replace: "Any one annotation point can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." with: "The resolved set of annotations for a schema component can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. Errata Section 7.8 newVariableInstance dfdl:newVariableInstance can be placed as an annotation on sequence, choice, and group references. Replace: "It is a schema definition error to have more than one newVariableInstances for the same variable at any given point in the document." with "The resolved set of annotations for any schema component can contain multiple dfdl:newVariableInstance annotations, but they must each refer to a different variable. It is a schema definition error otherwise." Errata Section 7.9: setVariable Correct language to be consistent with: "dfdl:setVariable may be placed as an annotation on sequence, choice, group references, local and global element declarations for elements of simple type, element references to elements of simple type, and simple type definitions. " Replace "It is a schema definition error to have more than one dfdl:setVariable for the same variable at any given point in the document." with "The resolved set of annotations for any schema component can contain multiple dfdl:setVariable annotations, but they must each refer to a different variable. It is a schema definition error otherwise." Errata: New Section 7.10: Evaluation Order for Statement Annotations Of the resolved set of annotations for a schema component, some are statement annotations and the order of their evaluation relative to the actual processing of the schema component itself (parsing or unparsing per its format annotations) is as given in the ordered lists below. Implementations are free to optimize by recognizing and executing discriminators or assertions with testKind='expression' earlier so long as the resulting behavior is consistent with what results from the this description: For elements: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. Processing of the element schema component itself (as per format annotations) 4. dfdl:setVariable(s) - See note below. 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) For sequences and choices: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. dfdl:newVariableInstance(s) 4. dfdl:setVariable(s) 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3 or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) 7. Processing of the sequence/choice component itself (as per format annotations) Note on Discriminators with testKind='expression' When parsing, an attempt to evaluate a discriminator must be made even if preceding statements or the parse of the schema component ended in a parse error. This is because a discriminator's expression could evaluate to true thereby resolving a point of uncertainty even if the complete parsing of the construct ultimately caused a parse error. Such discriminator evaluation has access to the DFDL Infoset of the attempted parse as it existed immediately before detecting the parse failure. Attempts to reference parts of the DFDL Infoset that do not exist are parse errors. It is also a parse error to read any variables defined by any dfdl:newVariableInstance statement evaluations that failed, or to read any variables set by any dfdl:setVariable statements that failed. Note on the dfdl:setVariable Statement for Elements Of the resolved set of annotations, the dfdl:setVariable statements for an element schema component (combined from an element reference, element declaration, and base and derived simpleType definitions) are executed after the parsing of the element. This implies that these variables are set after the evaluation of expressions corresponding to any computed DFDL format properties for the same schema component, and so the variables may not be referenced from expressions that compute DFDL properties. That is, if an expression is used to provide the value of a format property (such as dfdl:terminator, or dfdl:byteOrder), the evaluation of that format property expression occurs before any dfdl:setVariable annotation from the resolved set of annotations for that schema component are executed; hence, the expression providing the value of the format property may not reference the variable. Schema authors can insert sequences to provide more precise control over when variables are set. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Ok, third time's the charm.... I did look for minutes mentioning whether we decided to move when these were evaluated.... On Mon, Nov 19, 2012 at 2:57 PM, Steve Hanson <smh@uk.ibm.com> wrote:
Mike, discriminators and asserts with testKind 'expression' are *always*evaluated after the component's format annotation, the timing does not depend on the component kind. See your original write up.
Regards
Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> IBM SWG, Hursley, UK* **smh@uk.ibm.com* <smh@uk.ibm.com> tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: dfdl-wg@ogf.org Date: 19/11/2012 17:43 Subject: Re: [DFDL-WG] Action 186: Re: DFDL Statement Evaluation Timing (Assert, Discriminator, SetVariable, NewVariableInstance) ------------------------------
Easily fixed. Global replace done.
On Mon, Nov 19, 2012 at 12:36 PM, Steve Hanson <*smh@uk.ibm.com*<smh@uk.ibm.com>> wrote: Mike, I think we agreed on 'resolved set of annotations' instead of 'combined annotations' ?
------------------------------
I had the action to create the final errata wording for this item (Action 186).
Last action issue was to change wording to insure it was all dealing with combined annotations uniformly, and change "annotated construct" to "schema component".
I've done that, but also reorganized this as errata on specific sections of the spec consistent with the rest of our errata, and turned the 'ordering' section into numbered lists for clarity.
-------------------------------------------------------*
*
* Errata section 6.2. Clarification:* At any single annotation point of the schema, there can be only one format annotation (dfdl:format, dfdl:element, dfdl:sequence, dfdl:choice, dfdl:group, dfdl:simpleType). * * * Errata section 3. Glossary entries* * * * Glossary*: Annotation point: A location within a DFDL schema where DFDL annotation elements are allowed to appear. Note that annotation point is defined in section 6.2, but it belongs also in the glossary. * * * Glossary*: DFDL Statement annotations, or just DFDL Statements, are the annotation elements dfdl:assert, dfdl:discriminator, dfdl:setVariable, and dfdl:newVariableInstance. * * * Glossary*: DFDL Defining Annotations are the annotation elements dfdl:defineFormat, dfdl:defineVariable, and dfdl:defineEscapeScheme * * * Glossary*: DFDL Format Annotations are the annotation elements dfdl:format, dfdl:element, dfdl:simpleType, dfdl:group, dfdl:sequence, and dfdl:choice. * * * Glossary*: Remove existing definition for "Format Annotations" * * * Glossary*: Physical layer - revised to: "Physical Layer - A DFDL Schema adds DFDL annotations onto an XSDL language schema. The annotations describe the physical representation or physical layer of the data." * * * Glossary*: *Resolved set of annotations*: When DFDL annotations appear on a group reference and the sequence or choice of the referenced global group, or appear among an element reference, an element declaration, and its type definition, then they are combined together and the resulting set of annotations is referred to as the *resolved set of annotations* for the schema component. * * * Errata Section 7.3.1: * Remove "DFDL asserts can be placed on components within a DFDL model." as this is made more specifically subsequently. Replace "More than one dfdl:assert may be used at an annotation point. The dfdl:asserts will be evaluated in the order defined in the schema." with "If the resolved set of annotations for a schema component contain multiple dfdl:assert statements, then those with testKind='pattern' are executed before those with testKind='expression' (the default). However, within each group the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely. See Section REF: *Evaluation Order for Statement Annotations.*" Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. * * * Errata Section 7.3.1, and 7.4.1: Assert and Discriminator Placement* Correct description to be consistent with: dfdl:assert and dfdl:discriminator can be placed as annotations on sequence, choice, group references, local and global element declarations, element references, and simple type definitions. * * * Errata Section 7.4.1:* Replace: "Any one annotation point can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." with: "The resolved set of annotations for a schema component can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. * * * Errata Section 7.8 newVariableInstance* dfdl:newVariableInstance can be placed as an annotation on sequence, choice, and group references. Replace: "It is a schema definition error to have more than one newVariableInstances for the same variable at any given point in the document." with "The resolved set of annotations for any schema component can contain multiple dfdl:newVariableInstance annotations, but they must each refer to a different variable. It is a schema definition error otherwise." * * * Errata Section 7.9: setVariable * Correct language to be consistent with: "dfdl:setVariable may be placed as an annotation on sequence, choice, group references, local and global element declarations for elements of simple type, element references to elements of simple type, and simple type definitions. " Replace "It is a schema definition error to have more than one dfdl:setVariable for the same variable at any given point in the document." with "The resolved set of annotations for any schema component can contain multiple dfdl:setVariable annotations, but they must each refer to a different variable. It is a schema definition error otherwise." * * * Errata: New Section 7.10: Evaluation Order for Statement Annotations* Of the resolved set of annotations for a schema component, some are statement annotations and the order of their evaluation relative to the actual processing of the schema component itself (parsing or unparsing per its format annotations) is as given in the ordered lists below. Implementations are free to optimize by recognizing and executing discriminators or assertions with testKind='expression' earlier so long as the resulting behavior is consistent with what results from the this description: For elements: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. Processing of the element schema component itself (as per format annotations) 4. dfdl:setVariable(s) - See note below. 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) For sequences and choices: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. dfdl:newVariableInstance(s) 4. dfdl:setVariable(s) 5. Processing of the sequence/choice component itself (as per format annotations) 6. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, 4, or 5. See note below. 7. dfdl:assert(s) with testKind='expression' (parsing only) *Note on Discriminators with testKind='expression'* When parsing, an attempt to evaluate a discriminator must be made even if preceding statements or the parse of the schema component ended in a parse error. This is because a discriminator's expression could evaluate to true thereby resolving a point of uncertainty even if the complete parsing of the construct ultimately caused a parse error. Such discriminator evaluation has access to the DFDL Infoset of the attempted parse as it existed immediately before detecting the parse failure. Attempts to reference parts of the DFDL Infoset that do not exist are parse errors. It is also a parse error to read any variables defined by any dfdl:newVariableInstance statement evaluations that failed, or to read any variables set by any dfdl:setVariable statements that failed. * * * Note on the dfdl:setVariable Statement for Elements* Of the resolved set of annotations, the dfdl:setVariable statements for an element schema component (combined from an element reference, element declaration, and base and derived simpleType definitions) are executed after the parsing of the element. This is incontrast to the evaluation order for dfdl:setVariable statements on the resolved annotations of a sequence/choice, which are executed before the parsing of the sequence/choice. For elements, this implies that these variables are set after the evaluation of expressions corresponding to any computed DFDL format properties for that element, and so the variables may not be referenced from expressions that compute these DFDL properties. That is, if an expression is used to provide the value of a format property (such as dfdl:terminator, or dfdl:byteOrder), the evaluation of that format property expression occurs before any dfdl:setVariable annotation from the resolved set of annotations for that element are executed; hence, the expression providing the value of the format property may not reference the variable. Schema authors can insert sequences to provide more precise control over when variables are set.
Hi Mike I have incorporated all this into Errata v011. A couple of comments: - You use 'parse error' in a few places where correct term is 'processing error' - I think your new section 7.10 should be in section 9, as it is talking about order of execution and consequences of failure which are introduced in section 9. - Regarding "It is also a processing error to read any variables defined by any dfdl:newVariableInstance statement evaluations that failed, or to read any variables set by any dfdl:setVariable statements that failed. ", can you give a scenario where the processor can get into this situation? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: dfdl-wg@ogf.org Date: 19/11/2012 20:15 Subject: Re: [DFDL-WG] Action 186: Re: DFDL Statement Evaluation Timing (Assert, Discriminator, SetVariable, NewVariableInstance) Ok, third time's the charm.... I did look for minutes mentioning whether we decided to move when these were evaluated.... On Mon, Nov 19, 2012 at 2:57 PM, Steve Hanson <smh@uk.ibm.com> wrote: Mike, discriminators and asserts with testKind 'expression' are always evaluated after the component's format annotation, the timing does not depend on the component kind. See your original write up. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: dfdl-wg@ogf.org Date: 19/11/2012 17:43 Subject: Re: [DFDL-WG] Action 186: Re: DFDL Statement Evaluation Timing (Assert, Discriminator, SetVariable, NewVariableInstance) Easily fixed. Global replace done. On Mon, Nov 19, 2012 at 12:36 PM, Steve Hanson <smh@uk.ibm.com> wrote: Mike, I think we agreed on 'resolved set of annotations' instead of 'combined annotations' ? I had the action to create the final errata wording for this item (Action 186). Last action issue was to change wording to insure it was all dealing with combined annotations uniformly, and change "annotated construct" to "schema component". I've done that, but also reorganized this as errata on specific sections of the spec consistent with the rest of our errata, and turned the 'ordering' section into numbered lists for clarity. ------------------------------------------------------- Errata section 6.2. Clarification: At any single annotation point of the schema, there can be only one format annotation (dfdl:format, dfdl:element, dfdl:sequence, dfdl:choice, dfdl:group, dfdl:simpleType). Errata section 3. Glossary entries Glossary: Annotation point: A location within a DFDL schema where DFDL annotation elements are allowed to appear. Note that annotation point is defined in section 6.2, but it belongs also in the glossary. Glossary: DFDL Statement annotations, or just DFDL Statements, are the annotation elements dfdl:assert, dfdl:discriminator, dfdl:setVariable, and dfdl:newVariableInstance. Glossary: DFDL Defining Annotations are the annotation elements dfdl:defineFormat, dfdl:defineVariable, and dfdl:defineEscapeScheme Glossary: DFDL Format Annotations are the annotation elements dfdl:format, dfdl:element, dfdl:simpleType, dfdl:group, dfdl:sequence, and dfdl:choice. Glossary: Remove existing definition for "Format Annotations" Glossary: Physical layer - revised to: "Physical Layer - A DFDL Schema adds DFDL annotations onto an XSDL language schema. The annotations describe the physical representation or physical layer of the data." Glossary: Resolved set of annotations: When DFDL annotations appear on a group reference and the sequence or choice of the referenced global group, or appear among an element reference, an element declaration, and its type definition, then they are combined together and the resulting set of annotations is referred to as the resolved set of annotations for the schema component. Errata Section 7.3.1: Remove "DFDL asserts can be placed on components within a DFDL model." as this is made more specifically subsequently. Replace "More than one dfdl:assert may be used at an annotation point. The dfdl:asserts will be evaluated in the order defined in the schema." with "If the resolved set of annotations for a schema component contain multiple dfdl:assert statements, then those with testKind='pattern' are executed before those with testKind='expression' (the default). However, within each group the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely. See Section REF: Evaluation Order for Statement Annotations." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. Errata Section 7.3.1, and 7.4.1: Assert and Discriminator Placement Correct description to be consistent with: dfdl:assert and dfdl:discriminator can be placed as annotations on sequence, choice, group references, local and global element declarations, element references, and simple type definitions. Errata Section 7.4.1: Replace: "Any one annotation point can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." with: "The resolved set of annotations for a schema component can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. Errata Section 7.8 newVariableInstance dfdl:newVariableInstance can be placed as an annotation on sequence, choice, and group references. Replace: "It is a schema definition error to have more than one newVariableInstances for the same variable at any given point in the document." with "The resolved set of annotations for any schema component can contain multiple dfdl:newVariableInstance annotations, but they must each refer to a different variable. It is a schema definition error otherwise." Errata Section 7.9: setVariable Correct language to be consistent with: "dfdl:setVariable may be placed as an annotation on sequence, choice, group references, local and global element declarations for elements of simple type, element references to elements of simple type, and simple type definitions. " Replace "It is a schema definition error to have more than one dfdl:setVariable for the same variable at any given point in the document." with "The resolved set of annotations for any schema component can contain multiple dfdl:setVariable annotations, but they must each refer to a different variable. It is a schema definition error otherwise." Errata: New Section 7.10: Evaluation Order for Statement Annotations Of the resolved set of annotations for a schema component, some are statement annotations and the order of their evaluation relative to the actual processing of the schema component itself (parsing or unparsing per its format annotations) is as given in the ordered lists below. Implementations are free to optimize by recognizing and executing discriminators or assertions with testKind='expression' earlier so long as the resulting behavior is consistent with what results from the this description: For elements: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. Processing of the element schema component itself (as per format annotations) 4. dfdl:setVariable(s) - See note below. 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) For sequences and choices: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. dfdl:newVariableInstance(s) 4. dfdl:setVariable(s) 5. Processing of the sequence/choice component itself (as per format annotations) 6. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, 4, or 5. See note below. 7. dfdl:assert(s) with testKind='expression' (parsing only) Note on Discriminators with testKind='expression' When parsing, an attempt to evaluate a discriminator must be made even if preceding statements or the parse of the schema component ended in a parse error. This is because a discriminator's expression could evaluate to true thereby resolving a point of uncertainty even if the complete parsing of the construct ultimately caused a parse error. Such discriminator evaluation has access to the DFDL Infoset of the attempted parse as it existed immediately before detecting the parse failure. Attempts to reference parts of the DFDL Infoset that do not exist are parse errors. It is also a parse error to read any variables defined by any dfdl:newVariableInstance statement evaluations that failed, or to read any variables set by any dfdl:setVariable statements that failed. Note on the dfdl:setVariable Statement for Elements Of the resolved set of annotations, the dfdl:setVariable statements for an element schema component (combined from an element reference, element declaration, and base and derived simpleType definitions) are executed after the parsing of the element. This is incontrast to the evaluation order for dfdl:setVariable statements on the resolved annotations of a sequence/choice, which are executed before the parsing of the sequence/choice. For elements, this implies that these variables are set after the evaluation of expressions corresponding to any computed DFDL format properties for that element, and so the variables may not be referenced from expressions that compute these DFDL properties. That is, if an expression is used to provide the value of a format property (such as dfdl:terminator, or dfdl:byteOrder), the evaluation of that format property expression occurs before any dfdl:setVariable annotation from the resolved set of annotations for that element are executed; hence, the expression providing the value of the format property may not reference the variable. Schema authors can insert sequences to provide more precise control over when variables are set. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (2)
-
Mike Beckerle -
Steve Hanson