- dfdl-wg - lists.ogf.org

Cancelled: OGF DFDL Working Group Call
by Steve Hanson 29 May '18

29 May '18

1 0

Apache Daffodil (incubating) 2.1.0 released
by Mike Beckerle 16 May '18

16 May '18

The Apache Daffodil (incubating) community is pleased to announce the release of version 2.1.0. Release notes and downloads are available at: https://daffodil.apache.org/releases/2.1.0/ Daffodil is an open source implementation of the DFDL (Data Format Description Language) specification that uses DFDL schemas to parse fixed format data into an infoset, which is most commonly represented as either XML or JSON. This allows the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of the reverse by serializing or "unparsing" an XML or JSON infoset back to the original data format. For more information about Daffodil visit: https://daffodil.apache.org/ Regards, The Apache Daffodil Team Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

Action 304: Data Streaming (was) Re: Review/Feedback wanted on proposal: Data Streaming for base64 and other Layered Transformations
by Mike Beckerle 16 May '18

16 May '18

The data layering feature previously proposed has been implemented in Daffodil. There were minor changes from prior Wiki description. The wiki page is updated to reflect what the implementation actually does. The examples on the wiki page actually run and have been incorporated into Daffodil regression tests. https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Data+Layer… This feature is also being successfully used to create iCalendar and IMF format DFDL schemas. Those are not yet released for public consumption, but the feature is being successfully used for "real" formats and is thusfar, working as designed. Of the changes since prior draft of the design note, We changed a property value name. Property dfdl:layerLengthKind='boundaryMark' was done to replace the 'terminator' property value to avoid confusion with ordinary dfdl:terminator property. Thusfar we've not run into a need for a more general mechanism for passing parameters from the schema to the layering. The boundary mark, and the length have been sufficient, but this feels like an area where something more general may be needed in the future. For example, there are really 3 different kinds of layerings all known as "base64". Rather than having one general "base64" with a parameter, we're currently requiring that each variant have its own layering transform name, e.g., "base64_MIME" is one of them (the only one implemented thusfar). ...mikeb Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

Action 297 - places in spec for clarifications about delimited binary
by Mike Beckerle 16 May '18

16 May '18

I searched the spec for discussion of delimited binary, scanning, and escaping. I can find really only 2 places where a clarification is appropriate and would be helpful: The definition in the glosssary: *Delimiter scanning* - When parsing, the process of scanning for a specific item in the input data which either marks the end of an item or the beginning of a subsequent item. Delimiter scanning also takes into account escape schemes so as to allow the delimiters to appear within data if properly escaped. changes to (wording change in blue) *Delimiter scanning* - When parsing, the process of scanning for a specific item in the input data which either marks the end of an item or the beginning of a subsequent item. Delimiter scanning of text representations also takes into account escape schemes so as to allow the delimiters to appear within data if properly escaped. Section 12.3.2.2 Delimited Binary Data This section is extended to include: Delimited binary data may have initiators, terminators, and separators in any supported encoding however escape schemes are not available for delimited binary data. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

Tracker/Issue 233 - minor corrections
by Mike Beckerle 15 May '18

15 May '18

The phrase "When specified in a pattern, the 32-bit codepoint immediately following the pad escape is the pad character. " The term "32-bit codepoint" should be replaced by "character". In the phrase: For example, in the pattern "* #0 o''clock"... The font for the example pattern needs to be changed so that the two apostrophe/single-quotes after the lower case 'o' can be visually distinguished from the double quotes surrounding the example. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

Action 302 - backup of redmine issue trackers - 2018-05-15
by Mike Beckerle 15 May '18

15 May '18

I captured xhtml from atom feed for all open trackers which includes all the descriptions, and for each tracker with comments/updates, there is a separate xhtml page (from atom feed reader) with that content. I packaged it all in a zip file. I cannot attach the zip file of this content, as gmail doesn't allow zip file attachments. Here is a link to the file on google drive: https://drive.google.com/open?id=1tEQnr2HrC7_G6hmTPxSfGN6oCYdlBq2j I have added a calendar reminder so that every 3 months the DFDL Workgroup chairpersons will get a reminder to repeat this process. With that I believe we can close the action. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

Latest OGF DFDL WG Call Minutes
by Steve Hanson 15 May '18

15 May '18

Please find minutes from the latest call at https://redmine.ogf.org/projects/dfdl-wg/news Regards Steve Hanson IBM Hybrid Integration Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890

1 0

Typographical correction % -> %% - Section 31
by Mike Beckerle 15 May '18

15 May '18

Section 31.2, 31.3, 31.4 specify dfdl:escapeEscapeCharacter="%" This needs to be "%%". Or needs to be changed to use english prose to say the dfdl:escapeEscapeCharacter is '%' (requires property to be defined as dfdl:escapeEscapeCharacter="%%" since % are used for character entities syntax in DFDL string literals). Or perhaps better would be to change the usage of % in these examples to use $ or other neutral character to avoid the confusion. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

clarification needed: section 14.2.1 potentially trailing group
by Mike Beckerle 15 May '18

15 May '18

Section 14.2.1 introduces the term "potentially trailing group". It is referred to only in the next paragraph which defines "Trailing or Actually Trailing". Presumably this was done because it was intended that separator suppression would consider a potentially trailing group to have some separator suppression behavior. However, all subsequent discussion of separator suppression I find discusses only separators between elements. So we should either drop "potentially trailing group" and its definition, and modify the definition of "Trailing or Actually Trailing" to specify only elements.... or fix everyplace where things are specific to elements to refer to terms (term meaning element or model group). I suggest it is ok to eliminate "potentially trailing group". First, it simplifies things. If you want separator suppression behavior, well it applies only to potentially trailing things, and those things must be elements. Since only elements can be optional (only elements can have min/max occurs in DFDL), this means if you want separator suppression behavior, the separated things must be elements, period. Any model group, regardless of whether it has framing syntax, zero-length, whatever, would be considered required, and elements prior to it within a sequence group could not be considered potentially trailing and so separatorSuppressionPolicy would never result in suppression. I do not think we lose any expressive power in DFDL by requiring separator suppression to work only on elements. It perhaps requires you to put an element decl around something you might not want to. But there are lots of things in DFDL that require use of extra tiers of elements. Daffodil does not implement potentially trailing model groups, or if it does, it's super buggy. DFDL4S is binary only so doesn't have this stuff. What does IBM DFDL do here? Do you allow a sequence like: <sequence> optional element A optional element B <sequence> optional element C </sequence> optional element D </sequence> to have separator suppression that applies to elements A and B, or only to element D ? Comments? Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

Confirmed: OGF DFDL Working Group Call (15 May 15:00 GDT)
by Steve Hanson 15 May '18

15 May '18

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0