Re: [DFDL-WG] clarification needed: choice with direct dispatch is/is-not a PoU

17 Jul 2020

      Yes I think this clarification in 9.3.3.1 is definitely needed, as we
learned daffodil doesn't actually implement this right in the case of two
regular nested choices (where the inner one is NOT choice by dispatch).

Basically, there's a stack of discriminator booleans, and a discriminator
evaluating to true sets the state of the top of stack. That's wrong. It has
to find the first non-true discriminator, searching down the stack, and set
that one true instead.

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense |
www.owlcyberdefense.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
<http://www.ogf.org/About/abt_policies.php>

On Thu, Jul 16, 2020 at 3:09 AM Steve Hanson <smh@uk.ibm.com> wrote:
...
That's good to hear. I didn't think Daffodil had a different behaviour, as
I think it works with the IBM EDI schemas.
It is probably worth updating 9.3.3.1 to be explicit on what happens if a
second discriminator is encountered in a nested choice branch before a
further PoU is encountered.
Regards
Steve Hanson
IBM Hybrid Integration, Hursley, UK
Architect, *IBM DFDL*
<http://www.ibm.com/developerworks/library/se-dfdl/index.html>
Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/>
*smh@uk.ibm.com* <smh@uk.ibm.com>
tel:+44-1962-815848
mob:+44-7717-378890
Note: I work Tuesday to Friday
From:        Mike Beckerle <mbeckerle.dfdl@gmail.com>
To:        Steve Hanson <smh@uk.ibm.com>
Cc:        DFDL-WG <dfdl-wg@ogf.org>
Date:        15/07/2020 18:34
Subject:        [EXTERNAL] Re: [DFDL-WG] clarification needed: choice
with direct dispatch is/is-not a PoU
------------------------------
So, if the behavior is that after the choice-dispatch is complete, that a
discriminator on a branch resolves the outer choice, that is the semantics
I am also counting on working exactly that way.
If the existing documentation is consistent with that, then no change is
needed.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense |
*www.owlcyberdefense.com* <http://www.owlcyberdefense.com>
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the *OGF Intellectual Property Policy*
<http://www.ogf.org/About/abt_policies.php>
On Wed, Jul 15, 2020 at 5:08 AM Steve Hanson <*smh@uk.ibm.com*
<smh@uk.ibm.com>> wrote:
No. A discriminator is only ignored if there is no PoU in scope.
Otherwise it applies to the nearest in-scope PoU.  This is covered in
9.3.3.1 which deals with nested PoUs. It talks about the behaviour of a
processing error after a choice has been resolved. Given that an example of
a processing error is a discriminator resolving to false, the behaviour of
a discriminator evaluating to true is implied.
From the spec for direct dispatch choice ... *"**When a match is found,
it is as if a dfdl:discriminator had evaluated to true on that branch. It
is selected as resolution of the choice, and there is no backtracking to
try other alternative selections if a processing error occurs.**"*
So in your inner/outer scenario, if you encounter a further discriminator
on the resolved branch then that discriminates the OUTER choice.
The IBM schemas for EDI rely on this nested choice behaviour.  The inner
choice has a branch per possible transaction type, with a discriminator to
resolve each one. If a subsequent processing error occurs, it causes the
first branch of the OUTER choice to fail, which instead drives the 'Bad
Transaction' branch.  It would make no difference to the behaviour if the
inner choice was resolved by direct dispatch or initiatedContent.
<xsd:complexType name=*"Transaction"*>
      <xsd:sequence>
        <xsd:choice>
          <xsd:sequence>
            <xsd:choice>
              <xsd:element ref=*"v5010:T997"*>
                <xsd:annotation>
                  <xsd:appinfo source=*"**http://www.ogf.org/dfdl/*
<http://www.ogf.org/dfdl/>*"*>
                    <dfdl:discriminator>
{fn:contains(./ST/ST01_TransactionSetIdentifierCode,'997')}
                    </dfdl:discriminator>
                  </xsd:appinfo>
                </xsd:annotation>
              </xsd:element>
              <xsd:element ref=*"v5010:T998"*>
                <xsd:annotation>
                  <xsd:appinfo source=*"**http://www.ogf.org/dfdl/*
<http://www.ogf.org/dfdl/>*"*>
                    <dfdl:discriminator>
{fn:contains(./ST/ST01_TransactionSetIdentifierCode,'998')}
                    </dfdl:discriminator>
                  </xsd:appinfo>
                </xsd:annotation>
              </xsd:element>
              <xsd:sequence>
                <xsd:annotation>
                         <xsd:appinfo source=*"**http://www.ogf.org/dfdl/*
<http://www.ogf.org/dfdl/>*"*>
                               <dfdl:assert message=*"Unsupported
message"* test=*"{fn:false()}"*/>
                         </xsd:appinfo>
                </xsd:annotation>
              </xsd:sequence>
            </xsd:choice>
          </xsd:sequence>
          <xsd:sequence>
            <xsd:element ref=*"v5010:BadTransaction"*>
            </xsd:element>
          </xsd:sequence>
        </xsd:choice>
      </xsd:sequence>
    </xsd:complexType>
Bottom line is that you need to be careful with your discriminator
placement. Keep each discriminator as close as possible to the PoU it is
resolving.  You can always look down into structures.
Regards
Steve Hanson
IBM Hybrid Integration, Hursley, UK
Architect, *IBM DFDL*
<http://www.ibm.com/developerworks/library/se-dfdl/index.html>
Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/>
*smh@uk.ibm.com* <smh@uk.ibm.com>
tel:+44-1962-815848
mob:+44-7717-378890
Note: I work Tuesday to Friday
From:        Mike Beckerle <*mbeckerle.dfdl@gmail.com*
<mbeckerle.dfdl@gmail.com>>
To:        Steve Hanson <*smh@uk.ibm.com* <smh@uk.ibm.com>>
Cc:        DFDL-WG <*dfdl-wg@ogf.org* <dfdl-wg@ogf.org>>
Date:        14/07/2020 20:46
Subject:        [EXTERNAL] Re: [DFDL-WG] clarification needed: choice
with direct dispatch is/is-not a PoU
------------------------------
Ok, then to clarify, if I put a discriminator inside a branch of a choice
with direct dispatch, that discriminator should simply confirm the direct
dispatch selection of the choice dispatch key? I.e., it is ignored?
So if I have two nested choices, the outer backtracks, the inner is choice
by dispatch, then to discriminate the OUTER choice, I have to issue two
discriminators in a row. The first is a noop because it applies to the
inner choice. The second affects the outer choice?
This would seem to be the implications of having the choice with direct
dispatch be a PoU still.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense |
*www.owlcyberdefense.com* <http://www.owlcyberdefense.com>
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the *OGF Intellectual Property Policy*
<http://www.ogf.org/About/abt_policies.php>
On Tue, Jul 14, 2020 at 4:08 AM Steve Hanson <*smh@uk.ibm.com*
<smh@uk.ibm.com>> wrote:
I don't agree, because unlike an array with a fixed number of occurrences, it
is a *processing error* if the value of the expression does not match any
of the dfdl:choiceBranchKey property values for any of the branches. Which
currently causes backtracking because there is a PoU.
I consider direct dispatch as more like the use of dfdl:initiatedContent
when resolving a choice.
This is not a behaviour that can be changed in DFDL 1.0, it would affect
too many existing schemas. For example, IBM's DFDL schemas for SWIFT make
heavy use of direct dispatch.
Regards
Steve Hanson
IBM Hybrid Integration, Hursley, UK
Architect, *IBM DFDL*
<http://www.ibm.com/developerworks/library/se-dfdl/index.html>
Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/>
*smh@uk.ibm.com* <smh@uk.ibm.com>
tel:+44-1962-815848
mob:+44-7717-378890
Note: I work Tuesday to Friday
From:        Mike Beckerle <*mbeckerle.dfdl@gmail.com*
<mbeckerle.dfdl@gmail.com>>
To:        DFDL-WG <*dfdl-wg@ogf.org* <dfdl-wg@ogf.org>>
Date:        13/07/2020 14:50
Subject:        [EXTERNAL] [DFDL-WG] clarification needed: choice with
direct dispatch        is/is-not a PoU
Sent by:        "dfdl-wg" <*dfdl-wg-bounces@ogf.org*
<dfdl-wg-bounces@ogf.org>>
------------------------------
Just like an array with a computed number of occurrences, I believe a
choice with direct dispatch should have no PoU.
But the spec has this phrase "An xs:choice is always a point of
uncertainty. It is resolved sequentially, or by direct dispatch."
Which suggests there is a role for asserts/discriminators in resolving a
choice by direct dispatch even though there shouldn't be.
I think we should clarify this to "An xs:choice either is a point of
uncertainty, or uses direct dispatch."
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense |
*www.owlcyberdefense.com* <http://www.owlcyberdefense.com>
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the *OGF Intellectual Property Policy*
<http://www.ogf.org/About/abt_policies.php>
--
 dfdl-wg mailing list
 *dfdl-wg@ogf.org* <dfdl-wg@ogf.org>
 *https://www.ogf.org/mailman/listinfo/dfdl-wg*
<https://www.ogf.org/mailman/listinfo/dfdl-wg>
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU