August 2016 - dfdl-wg - lists.ogf.org

Spec bug: omission of mention of textBooleanJustification property
by Mike Beckerle 02 Aug '16

02 Aug '16

The sentence "The padding characters populate the *LeftPadding* and/or *RightPadding* regions depending on dfdl:textStringJustification, dfdl:textNumberJustification, or dfdl:textCalendarJustification, depending on the type of the element." Appears in the definition of textPadKind and textTrimKind. It omits the dfdl:textBooleanJustification case. The proper phrasing to match the wording of the prior sentence is "The padding characters populate the *LeftPadding* and/or *RightPadding* regions depending on dfdl:textStringJustification, dfdl:textNumberJustification, dfdl:textBooleanJustification, or dfdl:textCalendarJustification, depending on the type of the element." Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

unparsing with dfdl:setVariable referring to dfdl:outputValueCalc element
by Mike Beckerle 02 Aug '16

02 Aug '16

I somehow think we already discussed this, but am unable to find the thread. Suppose: <xs:element name="len" type="xs:int" dfdl:outputValueCalc="{ dfdl:valueLength(../data) + 10 }"/> <xs:sequence><xs:annotation><xs:appinfo ...> <dfdl:setVariable name="var">{ ../len }" /> </xs:appinfo></xs:annotation></xs:setVariable> <xs:element name="data" dfdl:length="{ $var }"> .... </xs:element> In the above, when unparsing, the output value calc for len can be evaluated, but we must delay its evaluation and unparsing until the subsequent data element is available. The next thing the unparse has to do, after delaying the unparsing of 'len' is set the var variable. This requires the value of len, which has been deferred. We have real schemas (e.g., even PCAP) where this occurs. This breaks the rule that when variables are evaluated, things they reference must have already been evaluated. Basically, when we delay evaluating the OVC for len we are suspending anything that depends on len having a value as well. Did we discuss and/or agree on this behavior? I don't have any work around for this. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

Need dfdl:inputValueCalc and dfdl:outputValueCalc on same element
by Mike Beckerle 02 Aug '16

02 Aug '16

So in PCAP format we find something like this: <element name="length" type="xs:int" dfdl:outputValueCalc="{ dfdl:contentLength(../payload) + 20 }"> <element name="payloadLength" type="xs:int" dfdl:inputValueCalc="{ ../length - 20 }" /> <element name="payload" dfdl:length="{ ../payloadLength }"> <complexType>.....</complexType> </element> To make this unparse we really need the payloadLength element to carry both an IVC and an OVC. Otherwise when unparsing the dfdl:length expression on the payload element can't be meaningful. We have an erratum that states that length expressions are always re-evaluated when unparsing, so as to produce a target length that can be used to insert padding/fillByte. However, the spec currently says you cannot have both IVC and OVC on the same element. I do not recall why we have this restriction, other than trying to prohibit things we didn't think were necessary, but it seems in this situation to be needed. Comments? Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 3

Re: [DFDL-WG] OGF DFDL WG Call Agenda 2016-08-02
by Steve Hanson 02 Aug '16

02 Aug '16

Mike I'm afraid that I got tied up in another meeting. I am free at 4pm UK. Please let me know if that works for you. Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org Cc: "Mike Beckerle" <mbeckerle(a)tresys.com>, "Michele Zundo" <michele.zundo(a)esa.int> Date: 01/08/2016 12:24 Subject: OGF DFDL WG Call Agenda 2016-08-02 Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13538?download= Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

OGF DFDL WG Call Agenda 2016-08-02
by Steve Hanson 01 Aug '16

01 Aug '16

Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13538?download= Regards Steve Hanson IBM Integration Bus, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890

1 0

Action 242: valueLength of complexType - contentLength and valueLength functions
by Mike Beckerle 01 Aug '16

01 Aug '16

(There are a few separate email threads related to Action 242. They have different subject lines, but all mention Action 242.) Experience with the Daffodil unparser implementation and dfdl:outputValueCalc has led us to a number of conclusions: 1) we need the concept of valueLength of a complexType which excludes the ElementUnused region, so that contentLength of a complexType can include the ElementUnused region. The example that motivates this is from PCAP where there are length fields that aren't the exact length while unparsing but contain other factors: <element name="len" type="xs:int" dfdl:outputValueCalc="{ fn:max(dfdl:valueLength(../ctElem, 'bytes'), 60) }" .../> ... <element name="ctElem" dfdl:length="{ ../len }" .../> In the above, we have a complex type element that has a minimum length of 60. So when unparsing if the actual length of the choice/sequence that is the model group of the ctElem is of length less than 60, we need to fill out the ElementUnused region of the DFDL grammar so as to achieve the min length of 60 bytes. If the dfdl:outputValueCalc of the len element was to call dfdl:contentLength, then we would have a circularity. Erratum 5.18 clarifies that length expressions are evaluated when unparsing to produce the target length that is used for pad/fill. This concept must be extended to complex types so that it also addresses the ElementUnused region that can follow the model group of a complex type. To clarify all this I suggest the DFDL data grammar must be changed as follows: ComplexNormalRep = LeftFraming PrefixLength ComplexContent RightFraming # Unchanged. Just here for context ComplexContent = ComplexValue ElementUnused ComplexValue = Sequence | Choice The above change basically moves ELementUnused out of the ComplexNormalRep production into a separate ComplexContent production, and distinguishes ComplexValue from ComplexContent. With the above grammar rule changes, I believe the spec as modified for Erratum 5.18 will be correct. (Everyplace where lengthKind explicit is lumped in with prefixed etc must be broken out, as it is a different case. ) 2) Spec says escapeCharacter, escapeBlockStart, and excapeBlockEnd contribute to the content length of an element. (Note: escapeEscapeCharacter is missing this stipulation and should be made consistent.) However, I believe this is incorrect. These all contribute to the *value* length, that is to say, they appear within the "SimpleValue | NilLogicalValue" region of the grammar. 3) Definition of dfdl:valueLength must be changed to specify that it returns the length of the ComplexValue region for complex types, and excludes any filling of the ElementUnused region. The definition of dfdl:contentLength is acceptable as is. However, both require clarification as to when "characters" and "bytes" are allowed relative to the lengthUnits of the element that is the argument. (This length units topic is a separate matter and will be a separate message.) Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1