- dfdl-wg - lists.ogf.org

OGF DFDL WG Call Agenda 2017-10-03
by Steve Hanson 03 Oct '17

03 Oct '17

Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13566?download= Regards Steve Hanson IBM Hybrid Integration Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890

1 0

Confirmed: OGF DFDL Working Group call (3 Oct 15:00 GDT in IBM: Hursley DE3S09)
by Steve Hanson 03 Oct '17

03 Oct '17

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

propose new dfdl:binaryNumberRep "offsetBinary"
by Mike Beckerle 15 Sep '17

15 Sep '17

We have found a number of places that use offset-binary numeric representation. This is also called excess-K, or biased, but I think offset binary is a better description of it. In this representation you take an unsigned binary, and just subtract an offset. E.g., for a 3-bit number, mostSignificantBitFirst: bits unsigned twos-comp offsetBinary 000 0 0 -4 001 1 1 -3 010 2 2 -2 011 3 3 -1 100 4 -4 0 101 5 -3 1 110 6 -2 2 111 7 -1 3 At the moment, users have to work around this in Daffodil using inputValueCalc and outputValueCalc. This is feasible, but really awkward for such a simple concept. So we suggest that the next revision of DFDL include dfdl:binaryNumberRep="offsetBinary" Thoughts? ...mikeb Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 2

Cancelled: OGF DFDL Working Group call
by Steve Hanson 15 Sep '17

15 Sep '17

1 0

Announcement: Daffodil 2.0.0 Final Release
by Mike Beckerle 06 Sep '17

06 Sep '17

We are pleased to announce the release of Daffodil 2.0.0! Downloads are available as zip, tar, and rpm at: https://opensource.ncsa.illinois.edu/projects/artifacts.php?key=DFDL&versio… Daffodil is the open source implementation of the Open Grid Forum's Data Format Description Language (DFDL) specification. http://www.ogf.org/dfdl Daffodil uses DFDL descriptions to parse data into an XML or JSON infoset for ingestion and validation. Daffodil can also perform the reverse, by serializing or "unparsing" an infoset back to the original data format. Daffodil 2.0.0 is a monumental milestone that denotes stability and usability, and is ready for general availability. Major features that make up this release include: - Support for unparse with one-to-one feature parity with parse - New API with native support for JDOM, w3c DOM, Scala XML, XML text, and JSON - Support for many non-byte size encodings - Support for xs:choice direct dispatch - Support for xs:boolean types - Support for xs:ignoreCase="yes" - Support for xs:union restrictions - Many improvements to the DFDL expression language - Numerous bug fixes and performance improvements More details of the changes are available: https://opensource.ncsa.illinois.edu/confluence/display/DFDL/Changelog https://opensource.ncsa.illinois.edu/jira/secure/ReleaseNote.jspa?projectId… Daffodil 2.0.0 has also been integrated into Apache NiFi and XML Calabash, with the source available in the following git repositories: Apache NiFi: https://opensource.ncsa.illinois.edu/bitbucket/projects/DFDL/repos/daffodil… XML Calabash: https://opensource.ncsa.illinois.edu/bitbucket/projects/DFDL/repos/daffodil… Although Daffodil has reached this milestone, active development continues, with some features not yet available. For a detailed list of unsupported features, visit the unsupported features and errata pages: https://opensource.ncsa.illinois.edu/confluence/display/DFDL/Unsupported+Fe… https://opensource.ncsa.illinois.edu/confluence/display/DFDL/Unsupported+Er… For more information on Daffodil, visit the Daffodil wiki: https://opensource.ncsa.illinois.edu/confluence/display/DFDL/ For questions and support, the development team is available on the Daffodil mailing list: http://oss.tresys.com/mailman/listinfo/daffodil-users as well as XMPP HipChat: http://hipchat.ncsa.illinois.edu/gvZdmJHmq If you are interested in becoming a Daffodil contributor, visit the daffodil wiki or the developers mailing list: https://opensource.ncsa.illinois.edu/confluence/display/DFDL/ http://oss.tresys.com/mailman/listinfo/daffodil-dev For a complete list of all open tickets, visit the Daffodil bug tracker: https://opensource.ncsa.illinois.edu/jira/browse/DFDL/ Thanks to everyone that has helped to make this milestone possible! Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

1 0

Cancelled: OGF DFDL Working Group call
by Steve Hanson 04 Sep '17

04 Sep '17

1 0

clarification - dfdl:choiceDispatchKey does not specify forward-reference restrictions
by Mike Beckerle 30 Aug '17

30 Aug '17

The test expressions in dfdl:assert and discriminator both specify that they may refer to the value of this element or its descendents. Hence, a test can refer to complex content down and inside the element. Most other properties can refer only backward, and not to the value of the element on which they appear (as their values are likely needed in order to parse that data and get an element at all.) However for dfdl:choiceDispatchKey, there is no description of what the expression may refer to. There is a common use case for choices which is where the fields(s) that the choiceDispatchKey wants to inquire about are found at a fixed position inside each of the choice branches. Programming languages use offsets to reach forward into the data to retrieve these. The only way this can be expressed in DFDL, without running into problems with polymorphic path steps, requires such elements to appear as children of a sequence that is the root of each branch. So for example: <element name="msg"> <complexType> <choice dfdl:choiceDispatchKey="{ ../msg/key }"> <sequence dfdl:choiceBranchKey="1"> ... stuff ... <element name="key" ..../> ... more stuff... </sequence> <sequence dfdl:choiceBranchKey="3"> ... stuff ... <element name="key" ..../> ... more stuff... </sequence> .... and so on ... </choice> </complexType> </element> This uses paths that reach down into the inside of each branch to obtain the key element. So is this legal, or not? Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

Cancelled: OGF DFDL Working Group call
by Steve Hanson 18 Aug '17

18 Aug '17

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

clarification question on terminators vs. enclosing group separators/terminators
by Mike Beckerle 17 Aug '17

17 Aug '17

The DFDL Spec says: 12.3.2 dfdl:lengthKind 'delimited' On parsing, the length of an element with dfdl:lengthKind 'delimited' is determined by scanning the datastream for the delimiter. The data stream is scanned for any of · the element's terminator (if specified) · an enclosing construct's separator or terminator · the end of an enclosing element designated by its known length · the end of the data stream So if an element has a terminator, are the enclosing constructs' separator or terminator also relevant? Or is ONLY the element's own terminator relevant for scanning, and hence, only the element's own terminator must be escaped if it appears in the content. For example, in a space-separated group, an enclosed element has a terminator ";". When parsing that element, do spaces have to be escaped if they appear in the content, or does only the terminator ";" have to be escaped? Strictly speaking it seems enclosing delimiters shouldn't have to be escaped, because the data must have the ";", and spaces are only significant as separators after finding the ";" terminator. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 3

OGF DFDL WG Call 2017-08-08 - cancelled
by Steve Hanson 09 Aug '17

09 Aug '17

Steve not available at short notice so cancelling. Apologies. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0