[DFDL-WG] Action 289 - example of backward reference to OVC element (from PCAP)

14 May 2018

      This is taken from the PCAP schema on DFDLSchemas.

However, the element OrigLen trivially refers to the prior element InclLen,
using dfdl:outputValueCalc. These two fields hold the same value.

(There must be a case where they don't, but we're not yet modeling that
case in this schema.)

     <xs:element name="Packet" maxOccurs="unbounded">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="PacketHeader">
                <xs:complexType>
                  <xs:sequence>
                    <xs:element name="Seconds" type="pcap:uint32"/>
                    <xs:element name="USeconds" type="pcap:uint32"/>
                    <xs:element name="InclLen" type="pcap:uint32"
                       dfdl:outputValueCalc="{ if
(dfdl:valueLength(../../pcap:LinkLayer/pcap:Ethernet, 'bytes') le 60) then
60 else dfdl:valueLength(../../pcap:LinkLayer/pcap:Ethernet, 'bytes') }"/>
                    </xs:sequence>
                    <xs:element name="OrigLen" type="pcap:uint32"
                       dfdl:outputValueCalc="{ ../InclLen }"/>
                </xs:complexType>
              </xs:element>
              <xs:element ref="pcap:LinkLayer" dfdl:lengthUnits="bytes"
dfdl:lengthKind="explicit" dfdl:length="{ ../PacketHeader/InclLen }"/>
            </xs:sequence>
          </xs:complexType>
        </xs:element>

It's not a very interesting case, but it is one where an element refers
backward to a prior element from an OVC, where that element is also not
computed because it is an OVC that is referencing forward.

A related issue that may require a separate action item is this:

Here's the same DFDL schema fragment, enhanced to avoid the redundant
subexpression in the InclLen outputValueCalc:

    <xs:element name="Packet" maxOccurs="unbounded">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="PacketHeader">
                <xs:complexType>
                  <xs:sequence>
                    <xs:element name="Seconds" type="pcap:uint32"/>
                    <xs:element name="USeconds" type="pcap:uint32"/>
                    <xs:sequence>
                    <xs:annotation><xs:appinfo source="
http://www.ogf.org/dfdl/">
                      <dfdl:newVariableInstance
ref="tns:ethernetValueLength"
                        value="{
dfdl:valueLength(../../pcap:LinkLayer/pcap:Ethernet, 'bytes') } "
                    </xs:appinfo></xs:annotation>
                    <xs:element name="InclLen" type="pcap:uint32"
                       dfdl:outputValueCalc="{ if ($tns:ethernetValueLength
le 60) then 60 else $tns:ethernetValueLength }"/>
                    </xs:sequence>
                    <xs:element name="OrigLen" type="pcap:uint32"
                       dfdl:outputValueCalc="{ ../InclLen }"/>
                  </xs:sequence>
                </xs:complexType>
              </xs:element>
              <xs:element ref="pcap:LinkLayer" dfdl:lengthUnits="bytes"
dfdl:lengthKind="explicit" dfdl:length="{ ../PacketHeader/InclLen }"/>
            </xs:sequence>
          </xs:complexType>
        </xs:element>

Now consider when parsing, this dfdl:newVariableInstance statement will be
encountered, and there is no information to tell the parser that this
variable is not needed at parse time.
So it will attempt to evaluate the expression, and that expression will not
be able to be evaluated, because it is forward referencing.

Without dfdl:newVariableInstance, one can accomplish the same thing by
using a hidden group with a zero-length element having
dfdl:outputValueCalc. However, note in that case we know, by the
dfdl:outputValueCalc, that the computation is ONLY needed when unparsing.
With dfdl:newVariableInstance, there is no such knowledge.

I think a parse/unparse flag is needed for dfdl:newVariableInstance to
prevent the parser from encountering an error.  It is quite possible that
at parse time there will be analogous newVariableInstance expressions only
needed at parse time. However, as there cannot be forward references at
parse time, those expressions have to work. So the flag is *in principle*
only needed to tell the parser to ignore the expression because it is only
used for unparsing.

Here's another example of the same phenominon. This group is hidden and
appears before the ../IPSrc it refers to. In the Infoset, ../IPSrc will
contain a typical IPAddress string e.g., 10.20.30.40. This group picks that
string apart at the "." dots into 4 binary bytes which are the
representation.

<xs:group name="IPSrcGrp">
    <xs:sequence>
        <xs:annotation><xs:appinfo="http://www.ogf.org/dfdl/">
          <dfdl:newVariableInstance ref="tns:afterIPSRCByte1"
             value="{ fn:substring-after(../IPSrc, '.') "/>
          <dfdl:newVariableInstance ref="tns:afterIPSRCByte2"
             value="{ fn:substring-after($tns:afterIPSRCByte1, '.')}" />
          <dfdl:newVariableInstance ref="tns:afterIPSRCByte3"
             value="{ fn:substring-after($tns:afterIPSRCByte2, '.')}" />
        </xs:appinfo></xs:annotation>
        <xs:element name="IPSrcByte1" type="xs:unsignedByte"
dfdl:outputValueCalc="{
         xs:unsignedByte(fn:substring-before(../IPSrc, '.'))
         }"/>
        <xs:element name="IPSrcByte2" type="xs:unsignedByte"
dfdl:outputValueCalc="{
         xs:unsignedByte(fn:substring-before($tns:afterIPSRCByte1, '.'))
         }"/>
        <xs:element name="IPSrcByte3" type="xs:unsignedByte"
dfdl:outputValueCalc="{
         xs:unsignedByte(fn:substring-before($tns:afterIPSRCByte2, '.'))
         }"/>
        <xs:element name="IPSrcByte4" type="xs:unsignedByte"
dfdl:outputValueCalc="{
         xs:unsignedByte($tns:tns:afterIPSRCByte3)
         }"/>
    </xs:sequence>
  </xs:group>

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
<http://www.ogf.org/About/abt_policies.php>