A point of uncertainty occurs when parsing a schema component when an occurrence of that schema component might not be the next item encountered in the data stream. Points of uncertainty can be nested.

Any one of the following schema constructs is a potential point of uncertainty:

·        A branch of xs:choice

·        All xs:elements in an unordered xs:sequence (dfdl:sequenceKind is 'unordered')

·        An optional xs:element

·        An array xs:element.

·        All xs:elements in an xs:sequence containing one or more floating xs:elements.

The parser resolves these points of uncertainty by way of a set of construct-specific rules given below along with determining whether schema components are known-to-exist or known-not-to-exist. For some of these constructs, there are situations where while there is the potential for uncertainty, the circumstances are such that there isn't any actual uncertainty; hence, potential points of uncertainty are distinguished from actual points of uncertainty below.
A branch of xs:choice is always an actual point of uncertainty. A choice is resolved sequentially, or by direct dispatch. Sequential choice resolution occurs by parsing each choice branch in schema definition order until one is known-to-exist. It is a processing error if none of the choice branches are known-to-exist. Direct-dispatch choice resolution occurs by matching the value of the dfdl:choiceDispatchKey property to the value of the dfdl:choiceChoiceBranchKey property of one of the choice branches. It is a processing error if none of the choice branches have a matching value in their dfdl:choiceChoiceBranchKey property.

An element in an unordered xs:sequence is always an actual point of uncertainty. It is resolved by parsing for the child components of the sequence in schema definition order at each point in the data stream where a component can exist until the required number of occurrences of each child component is known- to-exist or the sequence is terminated by delimiters or specified length.

An element in a sequence with one or more floating elements is always an actual point of uncertainty. It is resolved by parsing for the expected element at that point in the data stream. If the expected element is known-not-to-exist then an occurrence of each floating element is parsed in schema definition order.

When parsing an array, points of uncertainty only occur for certain values of occursCountKind, as follows:

occursCountKind	Details of Potential and Actual Points of Uncertainty
fixed	No potential point of uncertainty (maxOccurs occurrences expected).
implicit	All ocurrences are potential points of uncertainty. An actual point of uncertainty exists after minOccurs occurrences found and until maxOccurs occurrences have been found.
parsed	All occurrences are actual points of uncertainty.
expression	No potential point of uncertainty (dfdl:occursCount occurrences expected)
stopValue	No potential point of uncertainty (the stopValue must always be present, even when minOccurs is 0).

Table 11: Points of Uncertainty and dfdl:occursCountKind

An optional element point of uncertainty is resolved by parsing the element until it is either known-to-exist or known-not-to-exist. Whether an optional element is an actual point of uncertainty depends on property dfdl:occursCountKind as described above. (Property dfdl:occursCountKind is defined in Section 16.1 dfdl:occursCountKind property.)

For an array element, the point of uncertainty is resolved for each occurrence separately by parsing the occurrence until it is either known-to-exist or known-not-to-exist.

Discriminators resolve potential points of uncertainty. A discriminator defined on, or contained by, a schema construct that is a potential point of uncertainty, will only ever resolve that point of uncertainty. This holds regardless of whether there is any actual uncertainty.
For example, if a discriminator is defined on an array element which is contained within the branch of a choice, the discriminator will only resolve the existence of occurrences of the array element, and never the existence of the occurrence of the choice branch. As another example, consider an array element with dfdl:occursCountKind 'implicit' and minOccurs '1'. The first element of such an array must exist, so there is no actual uncertainty. A discriminator on such an element is redundant, but often must be expressed so as to discriminate the existence of the second and any subsequent array elements. If a discriminator evaluates to 'false' or causes a processing error on a potential point of uncertainty where there is no actual uncertainty, ..... TBD

 
(I think this causes a processing error which will fail the whole array.....but that sounds like it contradicts the statement above that says "it only ever resolves that point of uncertainty" ?)

------------------------------------

7.4        The dfdl:discriminator Statement Annotation Element

DFDL discriminators are used during parsing to resolve points of uncertainty that cannot be resolved by speculative parsing. Discriminators are not used during unparsing.  They can also be used to force a resolution earlier during the parsing of a group so that subsequent parsing errors are treated as processing errors of a known component rather than a failure to find a component.

A discriminator determines the existence or non-existence of a component. If the discriminator is successful then the component is known to exist and any subsequent errors will not cause backtracking at points of uncertainty. If a discriminator is unsuccessful then the component is known not to exist and backtracking occurs immediately.

If the complex type of an element contains a sequence group as its content model then if the sequence group is known not to exist, then the element is known not to exist.

Examples of dfdl:discriminator annotation are below :

<dfdl:discriminator>
  { ../recType eq 0 }
</dfdl:discriminator>

<dfdl:discriminator test="{ ../recType eq 0}" />

When the discriminator's expression evaluates to "false", then it causes a processing error, and the discriminator is said to fail.

A discriminator defined on, or contained by, a schema construct that is a potential point of uncertainty, will only ever resolve that point of uncertainty.


Regards
 
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
----- Forwarded by Steve Hanson/UK/IBM on 17/11/2014 11:54 -----

From:        Steve Hanson/UK/IBM
To:        Tim Kimber/UK/IBM@IBMGB
Cc:        dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org
Date:        15/05/2014 10:48
Subject:        Re: [DFDL-WG] Fw: Action 248 (was Thoughts on a discriminator        scenario)

Based on the above, which reflects the email discussion below, here is what I propose to resolve this action.
1.        If a potential point of uncertainty is always an actual point of uncertainty (choice branch, element in unordered sequence, floating element, ock 'parsed') then a discriminator that applies to it will only ever resolve that point of uncertainty. It never has an effect on any enclosing point of uncertainty.  
2.        If a potential point of uncertainty is sometimes an actual point of uncertainty (ock 'implicit') then a discriminator that applies it will only ever resolve, or have no effect on, that point of uncertainty. It never has an effect on any enclosing point of uncertainty.
3.        If a potential point of uncertainty is never an actual point of uncertainty (ock 'fixed', 'expression', 'stopValue') then a discriminator that applies to it will never have an effect on that point of uncertainty. Nor does it ever have an effect on any enclosing point of uncertainty.
I think 1 and 2 are not controversial, but there is an alternative for 3:

 3.   If a potential point of uncertainty is never an actual point of uncertainty (ock 'fixed', 'expression', 'stopValue') then a discriminator that applies to it will never have an effect on that point of uncertainty. Instead the discriminator is applied to any enclosing point of uncertainty.

The alternative means that changing an element from (say) ock 'parsed' to ock 'expression' has the same effect on a discriminator as changing the element to (1,1). The discriminator that applied to it now applies to any enclosing pou.

SMH: Afternote: The alternative does not work for the reason given in my reply to Tim above.

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848



From:        Steve Hanson/UK/IBM
To:        Tim Kimber/UK/IBM@IBMGB,
Cc:        dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org
Date:        05/02/2014 12:04
Subject:        Re: [DFDL-WG] Action 248 (was Thoughts on a discriminator scenario)

I started on this by reading section 9.3.3 on points of uncertainty, which lists the potential PoUs. Here's the list to save getting the spec out.

1.        An xs:choice branch

2.        All xs:elements in an unordered xs:sequence (dfdl:sequenceKind is 'unordered')

3.        An optional xs:element

4.        An array xs:element

5.        All xs:elements in an xs:sequence containing one or more floating xs:elements.

The section then looks at each in turn and gives the circumstances when it is an actual PoU or not. As currently written, it is only 3 and 4 where a potential PoU might not be an actual PoU. For 1, 2 and 5 it says they are always actual PoUs.

But I'm not sure that's correct. A deeper analysis of what is actually going on with 1, 2 and 5 says to me that there are times when there might not be an actual PoU.

1. Given that there is no concept in DFDL of optional choice branches, then if the last branch is reached then there is no longer a PoU. It must be that branch else it is a processing error.

TK: I think of it slightly differently. It is a PoU, even if the branch is the only remaining branch. If we say that the final choice branch is not a PoU then diagnostics become confused - the parser reports the error code as 'error while parsing root/choice/lastBranch/field1' when the correct error code would be 'none of the branches of root/choice were found in the data'.

SMH: I see your point. My thinking was that choices have finite branches and a choice is (1,1). If I have got to the last branch then I am not one of the other branches so I must be this one. If there is any other possibility then the model is missing a branch, even if it is just one that contains an empty sequence with an assert {fn:false()}. In practice of course users forget to add that last branch (there's no XSDL equivalent to the 'default' branch of a switch/case statement), so yes they could end up with an unclear diagnostic.

2. There can come a point in an unordered sequence when all that can be encountered is one element, and if that is (1,1) then there is no longer a PoU.

TK: It's still a PoU. The specification says that occursCountKind is 'parsed' for all members of an unordered group, so min/maxOccurs do not come into play.

SMH: Interesting. The spec says that if a member is optional or an array then it must be 'parsed'. If it is (1,1) though it does not have an occursCountKind. The specific case I was thinking of is when all members are (1,1), so when you have one element to go there is no PoU. However, the rewrite into a repeating choice has the effect of making everything 'parsed', which is really the point you are making. So I agree with you, it is easier to say that everything is an actual PoU else it complicates the rewrite semantic.

5. If all floating elements are (1,1) and all are encountered, then from that point on there are no longer any PoUs due to floating elements.

TK: I suspect that floating elements are somewhat like unordered branches - most users will not want min/maxOccurs to affect the parsing of the group. Schema validation ( or more complex validation applied in the receiving application ) will deal with non-conformances.

SMH: Possibly yes. With something like X12 NTE segments, that is the case. But we don't express the floating semantic as a rewrite of the whole sequence like we do for unordered, it's more of a per element thing. And if that is done dynamically as we go through the sequence, having no PoU can result.  

I'd like us to get straight on this before I proceed with the action proper.

Regards

Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
----- Forwarded by Steve Hanson/UK/IBM on 05/02/2014 10:12 -----

From:        Steve Hanson/UK/IBM
To:        dfdl-wg@ogf.org,
Date:        27/01/2014 17:39
Subject:        Fw: Thoughts on a discriminator scenario

--
  dfdl-wg mailing list
  dfdl-wg@ogf.org
  https://www.ogf.org/mailman/listinfo/dfdl-wg


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU