- dfdl-wg - lists.ogf.org

Minutes from 2025-01-09 DFDL Workgroup Call
by Mike Beckerle 09 Jan '25

09 Jan '25

I've written these up here: https://github.com/OpenGridForum/DFDL/blob/master/calls/2025/2025-01-09_DFD… Comments and anything I missed welcome. You can edit the document, or reply to this email and I'll incorporate.

2 2

Clarification needed: multiple assert statements evaluation order
by Mike Beckerle 09 Jan '25

09 Jan '25

Given this DFDL annotation on an xs:sequence <xs:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:assert test="..." /> <dfdl:discriminator test="..." /> <dfdl:assert test="..." /> </xs:appinfo> The spec seems silent about the evaluation order among these 3 statement annotations. Let's call them A, B, C. First, the spec makes it clear that the discriminator B could be evaluated earlier than either assertion, and even before some of the sequence content. This is to allow optimization by a DFDL implementation. The spec is also clear that even if the parse of the sequence content fails, discriminator B is evaluated (with the infoset being the state at the time of the failure). Again this is to ensure the behavior matches that of an optimizing DFDL implementation. But let's assume an implementation does no such optimization. I believe these evaluation orders are legal for the 3 statements after the sequence content has been parsed: A, B, C - if A fails, we do know B must still be evaluated. B, A, C - this is the minimum sort of hoist/optimization, doing the discriminator before the asserts. Questions: 1) Are any other orders of evaluation allowed? 2) If evaluation of A fails, do we still evaluate assertion C? (My hope is the answer here is no, because that allows consecutive asserts to build on each other's assumptions. But the spec is unclear.) 3) Can users depend on the failure of A to generate a message output? (ex: if the assert has a message attribute, can we state that this message will somehow be exhibited or logged by the implementation, unless the failure is suppressed by backtracking at a point of uncertainty) If A fails, the spec does say that B must still be evaluated. But if A fails, will C be evaluated? Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

2 3

Canceled event with note: DFDL Workgroup - DFDL Spec Issues and Actions @ Thu Dec 12, 2024 11am - 12pm (EST) (dfdl-wg@ogf.org)
by Mike Beckerle 09 Dec '24

09 Dec '24

This event has been canceled with a note: "Neither Steve Hanson nor Mike Beckerle are available for this meeting. Next call January 9th. " DFDL Workgroup - DFDL Spec Issues and Actions Thursday Dec 12, 2024 ⋅ 11am – 12pm Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer Mike Beckerle mbeckerle.dfdl(a)gmail.com Guests Mike Beckerle - organizer Mike Beckerle - optional smhdfdl(a)gmail.com - optional jens.jensen(a)stfc.ac.uk - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional wiloneil(a)uk.ibm.com - optional wolfgang.ziegler(a)z-rands.de - optional ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

Clarification needed on dfdl:decimalSigned
by Mike Beckerle 18 Nov '24

18 Nov '24

This property appears in exactly 3 places in the DFDL spec. * Section 13.5 where it is defined * Appendix G: 36.1 Parser - property precedence tables * Appendix G: 36.2 Unparser - property precedence tables. The definition in Section 13.5 says Indicates whether an xs:decimal element is signed. See 13.6.2 Converting logical numbers to/from text representation and 13.7.1 Converting Logical Numbers to/from Binary to see how this affects the presence of the sign in the data stream. 'yes' means that the xs:decimal element is signed 'no' means that the xs:decimal element is not signed Neither Section 13.6.2 nor 13.7.1 mention the decimalSigned property. We need to add language which clarifies this. I believe this is roughly what is needed: In Section 13.6.2, add this material at the start of the section: *The signed number types are byte, short, int, long, integer, float, double* *The unsigned number types have names beginning with "unsigned", as well as nonNegativeInteger.* *For type xs:decimal the decimal is considered signed if dfdl:decimalSigned="yes" and unsigned if dfdl:decimalSigned="no". * In Section 13.7.1 we need to add this: *When dfdl:binaryNumberRep="binary", then dfdl:decimalSigned controls whether the underlying representation is interpreted as a twos-complement signed binary integer or an unsigned binary integer. This value is then scaled using dfdl:binaryDecimalVirtualPoint to convert to/from the infoset value. * *When dfdl:binaryNumberRep is 'bcd', no sign is possible and the dfdl:decimalSigned property is ignored. * *When dfdl:binaryNumberRep is 'packed' a sign nibble is required and the dfdl:decimalSigned property is ignored.* *(rationale: packed but unsigned is the same as BCD)* *When dfdl:binaryNumberRep is "ibm4690Packed" the dfdl:decimalSigned property is used to determine if negative values are allowed, i.e., the packed digits may be preceded by a negative sign nibble.* *If dfdl:decimalSigned is "no" then no sign nibble is expected or allowed. * Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

2 5

clarification needed: prefixLengthType has min/max facets prefixIncludesPrefixLength="yes"
by Mike Beckerle 14 Nov '24

14 Nov '24

(Note: email sent to dfdl-wg at lists.ogf.org, not just ogf.org per prior practice.) Sometime back, in order to allow facets to sanity check prefix length values, we specified that if facets are expressed on a prefixLengthType they would always be checked and if they do not hold it is a processing error. This correction was part of the most recent GFD.240 However, we did not specify if the facet check is done on the prefix length value before or after it is adjusted when prefixIncludesPrefixLength="yes". So for example, suppose there is a single-byte prefix on a string and dfdl:prefixIncludesPrefixLength is "yes". Then the minimum well-formed value of the prefix is "1" which indicates a length 0 string with a single byte prefix. Suppose I want to add XSD facets to sanity check this prefix value: <simpleType name="prefix1Byte"> <restriction type="xs:unsignedByte"> <maxInclusive value="99"/> </restriction> </simpleType> So, if this type is used as the dfdl:prefixLengthType of a string with dfdl:lengthKind='prefixed', and dfdl:prefixIncludesPrefixLength="yes" then does the check for 99 measure the length of the string, or the value of the prefix, which is the length of the string plus one? My thought is that there is a quasi-element corresponding to this single byte prefix. That quasi-element's value is what is constrained by this maxInclusive facet. So I'm claiming the facet value 99 should be compared against the stored value in the prefix quasi-element, that is before any value adjustment is done for dfdl:prefixIncludesPrefixLength. Which is to say, suppose you switched to dfdl:lengthKind="explicit", and made an actual element of the single unsiged byte "prefix" field. Then the adjusting for the length of that prefix length would be done in the dfdl:length expression for the string, and the dfdl:outputValueCalc for the single byte prefix field. Which is to say that facets constraining the value of that one-byte element would apply to the value directly, not after any adjustment. I think lengthKind prefixed should work exactly the same as when a real length element and outputValueCalc are used. One should not have to adjust facets on the type of the length field when when it is the type of the length element, no should one have to adjust the facets if you convert and use lengthKind 'prefixed' with the dfdl:prefixLengthType, the facets should mean the same thing. Any thoughts/comments? Note: daffodil PR about this: https://github.com/apache/daffodil/pull/1334 JIRA Issue: https://issues.apache.org/jira/browse/DAFFODIL-2660

2 2

Agenda for call on 2024-11-14
by Mike Beckerle 14 Nov '24

14 Nov '24

See: https://github.com/OpenGridForum/DFDL/blob/master/calls/2024/2024-11-14_DFD… Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

1 0

DFDL-WG: Call for Agenda Topics for Meeting/Call on Nov 14
by Mike Beckerle 07 Nov '24

07 Nov '24

Call for agenda items/topics for November Workgroup Call. I have two minor clarification issues that have come up in email: * decimalSigned * prefixLengthType Please send other agenda items you would like to discuss. Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

1 0

DFDL-WG Call/Meeting 2024-10-10 Agenda & Minutes
by Mike Beckerle 14 Oct '24

14 Oct '24

Due to mailing list bounces, this is going out after the fact. Apologies. Mail to dfdl-wg at lists.ogf.org appears to work properly. Here is the agenda and minutes for 2024-10-10 https://github.com/OpenGridForum/DFDL/blob/master/calls/2024/2024-10-10_DFD… Here is the minutes from 2024-09-12 Call: https://github.com/OpenGridForum/DFDL/blob/master/calls/2024/2024-09-12_DFD… Mike Beckerle OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Apache Daffodil PMC | daffodil.apache.org Owl Cyber Defense | www.owlcyberdefense.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

[ANNOUNCE] Apache Daffodil 3.8.0 Released
by Mike Beckerle 14 Jun '24

14 Jun '24

The Apache Daffodil community is pleased to announce the release of version 3.8.0. Notable changes in this release include a supported API for Daffodil layers. Layers are a Daffodil extension to the DFDL language which are small algorithmic code plugins for computing checksums/CRCs on regions of data, or for performing data encoding/decoding such as base64, line folding, or compress/decompress on regions of data. Detailed release notes and downloads are available at: https://daffodil.apache.org/releases/3.8.0/ Apache Daffodil is an open-source implementation of the DFDL specification ... y'all know this. https://daffodil.apache.org/ Regards, The Apache Daffodil Team

1 0

Canceled event with note: DFDL Workgroup - DFDL Spec Issues and Actions @ Thu Jul 11, 2024 11am - 12pm (EDT) (dfdl-wg@ogf.org)
by Mike Beckerle 13 Jun '24

13 Jun '24

This event has been canceled with a note: "Cancelled July meeting. Next meeting in August. " DFDL Workgroup - DFDL Spec Issues and Actions Thursday Jul 11, 2024 ⋅ 11am – 12pm Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer Mike Beckerle mbeckerle.dfdl(a)gmail.com Guests Mike Beckerle - organizer frostmar(a)uk.ibm.com Mike Beckerle - optional smhdfdl(a)gmail.com - optional jens.jensen(a)stfc.ac.uk - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional wiloneil(a)uk.ibm.com - optional wolfgang.ziegler(a)z-rands.de - optional ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. To stop receiving future updates for this event, decline this event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0