- dfdl-wg - lists.ogf.org

[ANNOUNCE] Apache Daffodil 3.3.0 Released
by Mike Beckerle 22 Mar '22

22 Mar '22

The Apache Daffodil community is pleased to announce the release of version 3.3.0. Notable changes in this release include bug fixes, improvements, and a new charset. Detailed release notes and downloads are available at: https://daffodil.apache.org/releases/3.3.0/ Apache Daffodil is an open-source implementation of the DFDL specification that uses DFDL data descriptions to parse fixed format data into an infoset. This infoset is commonly converted into XML or JSON to enable the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of serializing or "unparsing" data back to the original data format. The DFDL infoset can also be converted directly to/from the data structures carried by data processing frameworks so as to bypass any XML/JSON overheads. For more information about Daffodil visit: https://daffodil.apache.org/ Regards, The Apache Daffodil Team

1 0

DFDL as an ISO standard
by Steve Hanson 24 Feb '22

24 Feb '22

Alan, Wolfgang From the minutes of the last DFDL WG call in November, "OGF will write to ISO-IEC JST1 with the proposal" to make DFDL an ISO standard. Tomorrow is the first WG call since then, so just following up on that action. Regards Steve Hanson IBM Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com<mailto:smh@uk.ibm.com> tel:+44-7717-378890 Note: I work Tuesday to Friday -----Original Message----- mob:+44-7717-378890 Note: I work Tuesday to Friday ----- Forwarded by Steve Hanson/UK/IBM on 08/02/2022 16:27 ----- From: "Steve Hanson" <smh(a)uk.ibm.com> To: "Wolfgang Ziegler" <wolfgang.ziegler(a)z-rands.de> Cc: dfdl-wg(a)ogf.org, "Mike Beckerle" <mbeckerle(a)owlcyberdefense.com>, "DFDL Schemas for Commercial and Scientific Data Formats" <smhdfdl(a)gmail.com>, "Michele Zundo" <michele.zundo(a)esa.int>, "Liam O'Neill" <WILONEIL(a)uk.ibm.com>, "Sill, Alan" <Alan.Sill(a)ttu.edu> Date: 12/11/2021 13:19 Subject: [EXTERNAL] RE: [DFDL-WG] Agenda for OGF DFDL WG Call ________________________________ Hi Wolfgang Thanks for the links. This is also a useful link, as it shows the DMTF adoptions at a higher level -https://www.dmtf.org/about/register/apresourcesRegards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, ‍‍‍‍‍‍‍‍‍‍‍ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Hi Wolfgang Thanks for the links. This is also a useful link, as it shows the DMTF adoptions at a higher level - https://www.dmtf.org/about/register/apresources<https://www.dmtf.org/about/register/apresources> Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL<http://www.ibm.com/developerworks/library/se-dfdl/index.html> Co-Chair, OGF DFDL Working Group<http://www.ogf.org/dfdl/> smh(a)uk.ibm.com<mailto:smh@uk.ibm.com> tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday From: "Wolfgang Ziegler" <wolfgang.ziegler(a)z-rands.de> To: "Steve Hanson" <smh(a)uk.ibm.com>, dfdl-wg(a)ogf.org Cc: "Mike Beckerle" <mbeckerle(a)owlcyberdefense.com>, "DFDL Schemas for Commercial and Scientific Data Formats" <smhdfdl(a)gmail.com>, "Michele Zundo" <michele.zundo(a)esa.int>, "Liam O'Neill" <WILONEIL(a)uk.ibm.com>, "Sill, Alan" <Alan.Sill(a)ttu.edu> Date: 11/11/2021 17:00 Subject: [EXTERNAL] Re: [DFDL-WG] Agenda for OGF DFDL WG Call ________________________________ Dear Steve, dear Mike, the example for a standard developed outside of ISO and then published as an ISO standard is Open Virtualization Format Specification from DMTF https://www.dmtf.org/sites/default/files/standards/documents/DSP0243_1.1.0.…<https://www.dmtf.org/sites/default/files/standards/documents/DSP0243_1.1.0.…> as ISO standard https://www.iso.org/standard/72081.html<https://www.iso.org/standard/72081.html> You can find the description of the PAS process (among other things) here https://www.iec.ch/members_experts/refdocs/iec/Consolidated_JTC1_Supplement…<https://www.iec.ch/members_experts/refdocs/iec/Consolidated_JTC1_Supplement…> The WG5 chair is Fernando Gebara Filho <fgebara(a)amazon.com> Please send me a mail if you have more questions. Best regards Wolfgang Am 11.11.21 um 15:02 schrieb Steve Hanson: > Please find agenda for next call on GitHub > athttps://github.com/OpenGridForum/DFDL/blob/master/calls/2021-11-11_DFDL-W… > <https://github.com/OpenGridForum/DFDL/blob/master/calls/2021-11-11_DFDL-WG-…<https://github.com/OpenGridForum/DFDL/blob/master/calls/2021-11-11_DFDL-WG-…>> > > > Regards > Steve Hanson > > IBM Hybrid Integration, Hursley, UK > Architect, _IBM DFDL_ <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, _OGF DFDL Working Group_ <http://www.ogf.org/dfdl/<http://www.ogf.org/dfdl/>>_ > __smh(a)uk.ibm.com_ <mailto:smh@uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > Note: I work Tuesday to Friday > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > https://www.ogf.org/mailman/listinfo/dfdl-wg<https://www.ogf.org/mailman/listinfo/dfdl-wg> >

2 1

dfdl:prefixLengthType with dfdl:assert on it
by Mike Beckerle 17 Feb '22

17 Feb '22

I believe the DFDL spec does not specify whether the prefixLengthType for an element of lengthKind 'prefixed' can have assertions, and if those assertions are evaluated, and when. Consider this example below. We really want the assert on the bufferPayload type to be evaluated before the length computed from the prefix is used as a length because we'd like good diagnostic behavior that points out that the prefix value is way out of realistic range. In many formats we see 4-byte length prefixes, but the range of values for specific fields is relatively tiny, often even just 1 to 100. Enforcing this via the assert expressed on the prefixLengthType's simpleType definition makes this easy. I could go either way on whether a prefixLengthType can carry a discriminator. But an assert should definitely be allowed. ... <complexType> <sequence> <element name="payload" type="xs:hexBinary" dfdl:lengthKind="prefixed" dfdl:prefixIncludesPrefixLength="no" dfdl:prefixLengthType="p8:bufferPayload"/> </sequence> </complexType> ... <simpleType name="bufferPayload"> <annotation> <appinfo source="http://www.ogf.org/dfdl/"> <dfdl:assert>{ dfdl:checkConstraints(.) }</dfdl:assert> </appinfo> </annotation> <restriction base="p8:unsignedLong"> <minInclusive value="0"/> <maxInclusive value="32768"/> </restriction> </simpleType>

2 3

OGF DFDL Working Group Call
by Steve Hanson 08 Feb '22

08 Feb '22

Calls for 2022 OGF DFDL WG call dial-in details. https://ibm.webex.com/meet/smh OGF DFDL Home: http://www.ogf.org/dfdl GitHub DFDL: https://github.com/OpenGridForum/DFDL DFDL Getting Started: https://community.ibm.com/community/user/middleware/viewdocument/get-starte… DFDL 1.0 Spec: http://www.ogf.org/documents/GFD.207.pdf Daffodil Apache incubator: https://daffodil.apache.org/ IBM DFDL: https://www.ibm.com/support/knowledgecenter/en/SSTTDS_11.0.0/com.ibm.etools…

1 0

revisit: nillable complex types and %ES restriction
by Mike Beckerle 25 Jan '22

25 Jan '22

The Daffodil user community is complaining about the DFDL v1.0 restriction that nillable complex types can only have %ES; (empty data) as their nilled representation. Multiple important data standards find this restriction quite awkward. It has significant negative impact on usability because it eliminates polymorphism of paths. The path needed to refer to an element value has different ending element path steps depending on whether it is nilled or not. The value of having DFDL create an XML infoset is lost if something as basic as this cannot be done in such a way that path expressions work alike to how they would work in regular XML/XSD. Is there a significant reason why this restriction cannot be lifted? If not we would probably plan to define a dfdlx namespace property that can be used to enable this alternate behavior, and then we could prototype this new fuctionality in Daffodil and propose this for DFDL 2.0 revision.

2 1

clarification: lengthKind 'delimited' but no terminating delimiter in scope
by Mike Beckerle 25 Jan '22

25 Jan '22

Is this an SDE? Or allowed? Daffodil currently implements this as allowed, and it ends up behaving much like "endOfParent", i.e., a scan for delimiters, when none are specified, scans as far as it can. However, given that we have endOfParent to express this behavior, seems to me that delimited with no delimiter should be an SDE. Thoughts?

2 1

OGF DFDL WG Calls for 2022
by Steve Hanson 19 Jan '22

19 Jan '22

Due to an imminent change in my mail technology, I am deferring sending out calendar invites for our 2022 calls until February. I anticipate the first call will be 10th February. In 2021 we only had 7 calls, so for 2022 a call every 4 weeks would make sense. Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 Note: I work Tuesday to Friday

1 0

OGF DFDL WG Call 2021-12-09 cancelled
by Steve Hanson 03 Dec '21

03 Dec '21

Steve unable to make the call, due to a code freeze deadline. Next call will be in January, Steve will send out invites for 2022 before Christmas break. Happy Christmas all. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Cancelled: OGF DFDL Working Group Call
by Steve Hanson 03 Dec '21

03 Dec '21

1 0

What is Rationale why Nillable complex type elements can only have '%ES; ' as their dfdl:nilValue property
by Mike Beckerle 02 Dec '21

02 Dec '21

DFDL has this seemingly ad-hoc restriction. Users naturally want to model a complex element where "-" (dash) means the whole complex element is nilled, and if not "-" then we parse and produce a complex element. What is the rationale for this restriction?

2 3