- dfdl-wg - lists.ogf.org

DFDL working rev 1.2.0 - ISO Formatted but still "OGF content" version of Spec
by Mike Beckerle 02 Oct '25

02 Oct '25

This took a few hours, but worked. See: https://github.com/OpenGridForum/DFDL/blob/master/docs/draft-gwdrp-dfdl-v1.… There are also PDF and HTML versions of the same in the same folder. There's hidden text in the MS-Word ".docx" file explaining a few aspects of things. That hidden text doesn't show in the PDF nor HTML versions. This is an "OGF" document in that it has all the OGF specified sections, change history page, etc. They just use different fonts, A4 pages, and some OGF backmatter sections are relocated to the front matter to facilitate dual maintenance of an OGF-track and ISO-PAS submission flavor of this document. This is setup so making a new ISO submission from this requires changing the page headings/footings (while preserving their size so page breaks don't change), and replacing the front matter pages. Note: there are some ISO formatting things I chose not to do. One is changing all table captions to be at the top of the table. The example ISO standards document template is also uploaded so you can see the preferred style and ISO front matter. https://github.com/OpenGridForum/DFDL/blob/master/docs/Word_template_for_IS…

2 1

FYI: ASN.1 Encoding Control Notation (ECN) compared to DFDL
by Mike Beckerle 01 Oct '25

01 Oct '25

ASN.1 ECN has been an ISO standard thing for a while. ECN lets you add representation information to the ASN.1 schema, which sounds roughly DFDL-like. So now with DFDL being an ISO standard the obvious thing arises to compare/contrast them. So I did a quick exercise in that direction. An AI (with my help) created an equivalent ASN.1 Schema + ECN from the FakeTDL format specification text *document *on github. (FakeTDL is a small example of a fictitious military tactical data link on github DFDLSchemas) This PR exhibits that ASN.1 and ECN code and does some comparison. They’re not particularly equivalent – some limitations of ECN become apparent even in this small example. https://github.com/DFDLSchemas/faketdl/pull/12/files This is the chatGPT5 session I used to create the example (fairly long) https://chatgpt.com/share/68dd112a-3140-800f-ab7c-c0d043c0460a I asked it a bunch more things about ASN.1 ECN that go beyond just this example. By the end of this it was speculating too much, and confusing some concepts, but overall this was quite useful. Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

1 0

Synced invitation: DFDL Workgroup - DFDL Spec Issues and Actions @ Thu Aug 14, 2025 11am - 12pm (EDT) (dfdl-wg@ogf.org)
by mbeckerle.dfdl＠gmail.com 09 Sep '25

09 Sep '25

This email keeps the event up to date in your calendar. DFDL Workgroup - DFDL Spec Issues and Actions Thursday Aug 14, 2025 ⋅ 11am – 12pm Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer mbeckerle.dfdl(a)gmail.com mbeckerle.dfdl(a)gmail.com Guests mbeckerle.dfdl(a)gmail.com - organizer mbeckerle(a)gmail.com - optional smhdfdl(a)gmail.com - optional jens.jensen(a)stfc.ac.uk - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional wiloneil(a)uk.ibm.com - optional wolfgang.ziegler(a)z-rands.de - optional ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

Updated invitation with note: DFDL Workgroup - DFDL Spec Issues and Actions @ Monthly from 11am to 12pm on the second Thursday from Thu Oct 12, 2023 to Wed Sep 10 (EDT) (dfdl-wg@ogf.org)
by Mike Beckerle 09 Sep '25

09 Sep '25

This event has been updated with a note: "I am unable to meet Thursdays. Separate email thread will be about rescheduling. " Changed: time DFDL Workgroup - DFDL Spec Issues and Actions Monthly from 11am to 12pm on the second Thursday from Thursday Oct 12, 2023 to Wednesday Sep 10 Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer Mike Beckerle mbeckerle.dfdl(a)gmail.com Guests Mike Beckerle - organizer frostmar(a)uk.ibm.com Mike Beckerle - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional smhdfdl(a)gmail.com - optional wiloneil(a)uk.ibm.com - optional jens.jensen(a)stfc.ac.uk - optional wolfgang.ziegler(a)z-rands.de - optional View all guest info https://calendar.google.com/calendar/event?action=VIEW&eid=amJnNXNxYzdwZmlo… Reply for dfdl-wg(a)ogf.org and view more details https://calendar.google.com/calendar/event?action=VIEW&eid=amJnNXNxYzdwZmlo… Your attendance is optional. ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

Synced invitation: DFDL Workgroup - DFDL Spec Issues and Actions @ Monthly from 11am to 12pm on the second Thursday from Thu Oct 10, 2024 to Wed Sep 10 (EDT) (dfdl-wg@ogf.org)
by Mike Beckerle 09 Sep '25

09 Sep '25

This email keeps the event up to date in your calendar. DFDL Workgroup - DFDL Spec Issues and Actions Monthly from 11am to 12pm on the second Thursday from Thursday Oct 10, 2024 to Wednesday Sep 10 Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer Mike Beckerle mbeckerle.dfdl(a)gmail.com Guests Mike Beckerle - organizer Mike Beckerle - optional smhdfdl(a)gmail.com - optional jens.jensen(a)stfc.ac.uk - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional wiloneil(a)uk.ibm.com - optional wolfgang.ziegler(a)z-rands.de - optional ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

Canceled event with note: DFDL Workgroup - DFDL Spec Issues and Actions @ Thu Aug 14, 2025 11am - 12pm (EDT) (dfdl-wg@ogf.org)
by Mike Beckerle 12 Jun '25

12 Jun '25

This event has been canceled with a note: "Cancelling July and August calls for summer pause. " DFDL Workgroup - DFDL Spec Issues and Actions Thursday Aug 14, 2025 ⋅ 11am – 12pm Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer Mike Beckerle mbeckerle.dfdl(a)gmail.com Guests Mike Beckerle - organizer Mike Beckerle - optional smhdfdl(a)gmail.com - optional jens.jensen(a)stfc.ac.uk - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional wiloneil(a)uk.ibm.com - optional wolfgang.ziegler(a)z-rands.de - optional ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

Canceled event with note: DFDL Workgroup - DFDL Spec Issues and Actions @ Thu Jul 10, 2025 11am - 12pm (EDT) (dfdl-wg@ogf.org)
by Mike Beckerle 12 Jun '25

12 Jun '25

This event has been canceled with a note: "Cancelling July and August calls for summer pause. " DFDL Workgroup - DFDL Spec Issues and Actions Thursday Jul 10, 2025 ⋅ 11am – 12pm Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer Mike Beckerle mbeckerle.dfdl(a)gmail.com Guests Mike Beckerle - organizer Mike Beckerle - optional smhdfdl(a)gmail.com - optional jens.jensen(a)stfc.ac.uk - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional wiloneil(a)uk.ibm.com - optional wolfgang.ziegler(a)z-rands.de - optional ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

Summer pause to DFDL Workgroup Calls
by Mike Beckerle 12 Jun '25

12 Jun '25

I am cancelling the DFDL WG Calls for July and August since many people are away during those times. Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

1 0

Reminder: DFDL Workgroup call Thursday June 12, 11am EDT.US
by Mike Beckerle 09 Jun '25

09 Jun '25

This is a reminder that we're having a DFDL Workgroup call this Thursday June 11, 11am EDT.US. Please send items you'd like on the agenda, and let me know if you plan to attend. I have this topic for discussion: * Peek feature - related Issue 66 - https://github.com/OpenGridForum/DFDL/issues/66 (about a lookAhead function) An idea has been proposed to generalize this to a more flexible peek/look-ahead capability. This is a requested feature for DFDL 2.0 allowing a parser to learn from examining the data once, and then use that knowledge in a subsequent parse over the same data. Details: - annotation on a xs:sequence - a parser to speculate forward and when it reaches the end of the sequence, resetting the position to the start of the sequence, but retaining information from the parse. - the retained information certainly includes DFDL variables set during the peek-ahead parse. - more controversial is the ability to retain the infoset, effectively allowing one to parse the same data multiple times. - does nothing on unparse. The annotated sequence is simply skipped by the unparser. - seems very easy to implement. Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

1 0

Idea: Peek layer
by Mike Beckerle 02 Jun '25

02 Jun '25

I've been dealing with lots of data formats lately which involve lookahead needs, and I'm trying to come up with a cleaner, easier, and less ad-hoc way to deal with them in DFDL to propose for the future. I wanted to bounce this idea off the workgroup before prototyping. Idea: Property dfdlx:peek="yes/no". Property on a sequence model group. Compatible with a hidden group ref. Unlike other properties, which are disallowed on a sequence with dfdl:hiddenGroupRef, this could be allowed on hidden sequences. That's actually a primary use case for this. If "no", no behavior change (what things do now). If "yes", the parse happens, including set variable assignments, infoset creation, etc. Then at the end of the sequence if the parse is successful the position in the data stream is reset to where it was at the start of the peek sequence. If the parse fails you backtrack to the enclosing PoU as normal and everything about the peek (when inside the PoU) is discarded. This allows you to learn by parsing something in the data more than once. Once to discover something which goes into the parser infoset (hidden or not), and into single-assignments to DFDL variables. The second time can parse making use of this learning. This is sort of like backtracking at a PoU, but you don't undo anything except the position in the data stream. On unparsing, all data written while unparsing the infoset for a sequence with dfdl:peek="yes" is discarded. Or maybe we can just say the infoset corresponding to a sequence with dfdl:peek="yes" is not unparsed at all. Implementations could put a limit on how far ahead you can peek. But a minimum of say, 512 bytes or maybe a bit bigger makes sense I think. That would be enough for every use case I have. I believe current restrictions in DFDL to ensure forward progress when parsing are sufficient to make it impossible to delay parsing forever with this. I.e., parsing can take a long time, but it still has to terminate (at least in theory, if there is enough memory for a big infoset). I think this dfdlx:peek has some nice properties. Pro: This is the most important thing: No specialized constructs for looking ahead. Just use DFDL to learn about the data, save it in variables or a piece of infoset that you can navigate with expressions to utilize the knowledge. Pro: Composition properties are good. Nothing new to learn. I can think of no impact on backtracking or any other aspects. Pro: Pretty cheap to implement, so long as the amount you can peek ahead is reasonably bounded. Pro: Synergistic with existing things like newVariableInstance and hidden groups to capture learning from a peek ahead. Con: Huge generality. Peeking ahead with a sequence with really rich sub-structure, PoUs and backtracking inside it, etc. That's all enabled by this feature, but none of the use cases I have need anything like that level of generality. This is one of those things where the stuff people will invent with it are unanticipated. All thoughts / musings are welcome. Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

2 1