- dfdl-wg - lists.ogf.org

Call for Agenda: Call this week: Thurs April 10, 11am EDT.US
by Mike Beckerle 09 Apr '25

09 Apr '25

If there are specific agenda topics we should have the call, so let me know by Wed afternoon if you have a topic for discussion. Steve Hanson has already notified that he cannot attend this week's call. Despite my best efforts, I have not been able to make progress on action items for this workgroup this month, so I have nothing to report. (I am always so determined to make progress right after our calls, and then ... reality hits, and I get bogged down in other work.) Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

1 1

Canceled event with note: DFDL Workgroup - DFDL Spec Issues and Actions @ Thu Apr 10, 2025 11am - 12pm (EDT) (dfdl-wg@ogf.org)
by Mike Beckerle 09 Apr '25

09 Apr '25

This event has been canceled with a note: "April meeting cancelled. " DFDL Workgroup - DFDL Spec Issues and Actions Thursday Apr 10, 2025 ⋅ 11am – 12pm Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer Mike Beckerle mbeckerle.dfdl(a)gmail.com Guests Mike Beckerle - organizer Mike Beckerle - optional smhdfdl(a)gmail.com - optional jens.jensen(a)stfc.ac.uk - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional wiloneil(a)uk.ibm.com - optional wolfgang.ziegler(a)z-rands.de - optional ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

Reminder - DFDL Workgroup Call this week: Thursday March 13
by Mike Beckerle 11 Mar '25

11 Mar '25

This is a call for specific agenda/discussion topics. Note: US.EST has just changed to US.EDT, so we are now UTC-04:00 The call will be at 11am US.EDT

1 0

Fwd: from a daffodil DFDL discussion on AI/LLMs
by Mike Beckerle 21 Feb '25

21 Feb '25

Thought this might be of interest to the workgroup, or we might spawn a thread from this in the dfdl workgroup. ---------- Forwarded message --------- From: Mike Beckerle <mbeckerle(a)apache.org> Date: Fri, Feb 21, 2025 at 1:22 PM Subject: AI/LLMs - was: .... To: <users(a)daffodil.apache.org> Consistent with my experience. There's nowhere near enough DFDL materials on github or stack overflow (yet) for a LLM to really create DFDL from a spec, and I've had little luck uploading format specs and asking it questions about them. Just asking GPT4o about DFDL is quite interesting though. Here's a chat I did: https://chatgpt.com/share/67b8bf26-1814-800f-95d6-d08ac63e14b8 I actually tried to use GPT4o to help with generating a property index into the PDF so that one could quickly jump to properties. It can't even list the properties in DFDL, but asking this was very interesting though. DFDL has a property named "finalTerminatorCanBeMissing". If you read that once, grokked the concept, but couldn't remember the exact name, well "terminatorCanBeRequired" is not a bad guess, and that's what GPT4o's guess was, which shows that it does have this stuff organized somehow as concepts, not a bunch of snippets to regurgitate. When I followed up with: " I haven't heard of the dfdl:terminatorCanBeRequired before. How does that work?" it corrected its mistake, and did get the name dfdl:finalTerminatorCanBeMissing this time. So the challenge is, if you know DFDL well, you can spot its mistakes, but if you are a naive user trying to learn DFDL, it gets quite a bit right, but the mistakes it does make are perhaps problematic. I believe I can prompt GPT4.0 to create a DFDL schema for me. Since I know DFDL, this is not faster than typing for me. But if you don't know DFDL, you can ask it to do lots of things that would save time like 'move that local simple type definition to a separate file and name it "intType2"'. And it is going to do that sort of thing just fine, particularly because that is mostly just using knowledge of XSD. This is all pretty cool. On Sat, Feb 15, 2025 at 7:37 AM Claude Mamo <claude.mamo(a)gmail.com> wrote: > I don't know if it's possible to capture the DICOM format in DFDL > > > Hah, a research team generated DICOM DFDL schemas from LLMs: > > > https://github.com/narfindustries/llm-tests-langsec/blob/main/results/1.0/D… > > https://github.com/narfindustries/llm-tests-langsec/blob/main/results/1.0/D… > > https://github.com/narfindustries/llm-tests-langsec/blob/main/results/1.0/D… > > https://github.com/narfindustries/llm-tests-langsec/blob/main/results/1.0/D… > > https://github.com/narfindustries/llm-tests-langsec/blob/main/results/1.0/D… > > https://github.com/narfindustries/llm-tests-langsec/blob/main/results/1.0/D… > > https://github.com/narfindustries/llm-tests-langsec/blob/main/results/1.0/D… > > The schema from gemini particularly stands out ;) > > ... > >>

1 0

testing please ignore
by Mike Beckerle 13 Feb '25

13 Feb '25

testing if just dfdl-wg(a)ogf.org now works, or if the 'lists' part is needed as in dfdl-wg(a)lists.ogf.org.

1 0

Reminder: DFDL Workgroup Call this week: Thursday Jan 13. 11:00 - 12:00 EST.US (UTC-5)
by Mike Beckerle 13 Feb '25

13 Feb '25

Draft agenda here: https://github.com/OpenGridForum/DFDL/blob/master/calls/2025/2025-02-13_DFD… Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

2 3

Minutes from 2025-01-09 DFDL Workgroup Call
by Mike Beckerle 09 Jan '25

09 Jan '25

I've written these up here: https://github.com/OpenGridForum/DFDL/blob/master/calls/2025/2025-01-09_DFD… Comments and anything I missed welcome. You can edit the document, or reply to this email and I'll incorporate.

2 2

Clarification needed: multiple assert statements evaluation order
by Mike Beckerle 09 Jan '25

09 Jan '25

Given this DFDL annotation on an xs:sequence <xs:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:assert test="..." /> <dfdl:discriminator test="..." /> <dfdl:assert test="..." /> </xs:appinfo> The spec seems silent about the evaluation order among these 3 statement annotations. Let's call them A, B, C. First, the spec makes it clear that the discriminator B could be evaluated earlier than either assertion, and even before some of the sequence content. This is to allow optimization by a DFDL implementation. The spec is also clear that even if the parse of the sequence content fails, discriminator B is evaluated (with the infoset being the state at the time of the failure). Again this is to ensure the behavior matches that of an optimizing DFDL implementation. But let's assume an implementation does no such optimization. I believe these evaluation orders are legal for the 3 statements after the sequence content has been parsed: A, B, C - if A fails, we do know B must still be evaluated. B, A, C - this is the minimum sort of hoist/optimization, doing the discriminator before the asserts. Questions: 1) Are any other orders of evaluation allowed? 2) If evaluation of A fails, do we still evaluate assertion C? (My hope is the answer here is no, because that allows consecutive asserts to build on each other's assumptions. But the spec is unclear.) 3) Can users depend on the failure of A to generate a message output? (ex: if the assert has a message attribute, can we state that this message will somehow be exhibited or logged by the implementation, unless the failure is suppressed by backtracking at a point of uncertainty) If A fails, the spec does say that B must still be evaluated. But if A fails, will C be evaluated? Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

2 3

Canceled event with note: DFDL Workgroup - DFDL Spec Issues and Actions @ Thu Dec 12, 2024 11am - 12pm (EST) (dfdl-wg@ogf.org)
by Mike Beckerle 09 Dec '24

09 Dec '24

This event has been canceled with a note: "Neither Steve Hanson nor Mike Beckerle are available for this meeting. Next call January 9th. " DFDL Workgroup - DFDL Spec Issues and Actions Thursday Dec 12, 2024 ⋅ 11am – 12pm Eastern Time - New York Location Google Meet https://www.google.com/maps/search/Google+Meet?hl=en Join with Google Meet https://meet.google.com/ifg-hknz-ysw?hs=224 Please send email to open discussions and add agenda for this meeting. If there are no specific topics I will cancel the meeting. Organizer Mike Beckerle mbeckerle.dfdl(a)gmail.com Guests Mike Beckerle - organizer Mike Beckerle - optional smhdfdl(a)gmail.com - optional jens.jensen(a)stfc.ac.uk - optional alan.sill(a)ttu.edu - optional mbeckerle(a)owlcyberdefense.com - optional braddk(a)us.ibm.com - optional dfdl-wg(a)ogf.org - optional wiloneil(a)uk.ibm.com - optional wolfgang.ziegler(a)z-rands.de - optional ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

Clarification needed on dfdl:decimalSigned
by Mike Beckerle 18 Nov '24

18 Nov '24

This property appears in exactly 3 places in the DFDL spec. * Section 13.5 where it is defined * Appendix G: 36.1 Parser - property precedence tables * Appendix G: 36.2 Unparser - property precedence tables. The definition in Section 13.5 says Indicates whether an xs:decimal element is signed. See 13.6.2 Converting logical numbers to/from text representation and 13.7.1 Converting Logical Numbers to/from Binary to see how this affects the presence of the sign in the data stream. 'yes' means that the xs:decimal element is signed 'no' means that the xs:decimal element is not signed Neither Section 13.6.2 nor 13.7.1 mention the decimalSigned property. We need to add language which clarifies this. I believe this is roughly what is needed: In Section 13.6.2, add this material at the start of the section: *The signed number types are byte, short, int, long, integer, float, double* *The unsigned number types have names beginning with "unsigned", as well as nonNegativeInteger.* *For type xs:decimal the decimal is considered signed if dfdl:decimalSigned="yes" and unsigned if dfdl:decimalSigned="no". * In Section 13.7.1 we need to add this: *When dfdl:binaryNumberRep="binary", then dfdl:decimalSigned controls whether the underlying representation is interpreted as a twos-complement signed binary integer or an unsigned binary integer. This value is then scaled using dfdl:binaryDecimalVirtualPoint to convert to/from the infoset value. * *When dfdl:binaryNumberRep is 'bcd', no sign is possible and the dfdl:decimalSigned property is ignored. * *When dfdl:binaryNumberRep is 'packed' a sign nibble is required and the dfdl:decimalSigned property is ignored.* *(rationale: packed but unsigned is the same as BCD)* *When dfdl:binaryNumberRep is "ibm4690Packed" the dfdl:decimalSigned property is used to determine if negative values are allowed, i.e., the packed digits may be preceded by a negative sign nibble.* *If dfdl:decimalSigned is "no" then no sign nibble is expected or allowed. * Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

2 5