- dfdl-wg - lists.ogf.org

OGF DFDL Working Group Call
by Steve Hanson 05 Jan '23

05 Jan '23

Calls for 2023 OGF DFDL WG call dial-in details. https://ibm.webex.com/meet/smh OGF DFDL Home: http://www.ogf.org/dfdl GitHub DFDL: https://github.com/OpenGridForum/DFDL DFDL Getting Started: https://community.ibm.com/community/user/middleware/viewdocument/get-starte… DFDL 1.0 Spec: http://www.ogf.org/documents/GFD.207.pdf Daffodil Apache incubator: https://daffodil.apache.org/ IBM DFDL: https://www.ibm.com/support/knowledgecenter/en/SSTTDS_11.0.0/com.ibm.etools… Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

1 0

interesting format: RINEX
by Mike Beckerle 04 Jan '23

04 Jan '23

I ran into an interesting format today called RINEX that is problematic for DFDL v1.0: http://acc.igs.org/misc/rinex304.pdf This format has a multi-line header with various kinds of header lines which are 80 characters, but characters 1 to 60 are data, and 61 to 80 are the "label", and you need the label in order to know how to parse the data of each header line. This either requires deep (and slow) speculation, or some sort of look ahead feature. E.g., See page 69 of that spec document for an example. The headers appear not only at the start of the file, but at the start of each data block. We have run into the need for short, fixed-distance look ahead before in other formats. In the case of RINEX the lookahead is exactly 60 characters. In other formats it is also a very short distance, but one which is tricky to figure out as the spec doesn't say exactly how big the region to be looked past is. One would have to figure it out from the data format spec. It's always a constant of course. Mike Beckerle Apache Daffodil PMC | daffodil.apache.org OGF DFDL Workgroup Co-Chair | www.ogf.org/ogf/doku.php/standards/dfdl/dfdl Owl Cyber Defense | www.owlcyberdefense.com

2 2

Next OGF DFDL WG call - cancelled
by Steve Hanson 15 Dec '22

15 Dec '22

Mike is unable to make the next call on Oct 20th so cancelling. Next call is Nov 3rd. Regards Steve Hanson IBM Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com<mailto:smh@uk.ibm.com> tel:+44-7717-378890 Note: I work Tuesday to Friday Unless otherwise stated above: IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

1 1

Example announcement: "Joint Development Foundation recognized as an ISO/IEC JTC 1 PAS submitter and submits OpenChain for international review - Linux Foundation"
by Sill, Alan 04 Dec '22

04 Dec '22

DFDL folks, FYI, the following announcement from the Linux Foundation might be useful as an example of how we can go forward with announcements for DFDL as a submission to ISO/IEC JTC1 once you are ready to proceed. I suggest rather than waiting for a new version, you consider submitting DFDL 1.0 as an ISO standard and follow up with any updates as future submissions. Other approaches are possible, of course. Let us know if you need any assistance or would like to have further discussions on this process. Alan Alan Sill, Ph.D Managing Director, High Performance Computing Center Adjunct Professor of Physics and Astronomy Texas Tech University Drane 159, MS 4-1167 <http://www.hpcc.ttu.edu/>http://www.hpcc.ttu.edu<http://www.hpcc.ttu.edu/> Co-Director, NSF Center for Cloud and Autonomic Computing https://nsfcac.org<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnsfcac.or…> President, Open Grid Forum https://www.ogf.org e-mail: Alan.Sill(a)ttu.edu<mailto:Alan.Sill@ttu.edu> ph. 806-834-5940 ——— Joint Development Foundation recognized as an ISO/IEC JTC 1 PAS submitter and submits OpenChain for international review THE LINUX FOUNDATION | 12 MAY 2020 […] This week, we are proud to announce that the Joint Development Foundation<http://www.jointdevelopment.org/> (JDF), which became part of the Linux Foundation family in 2019, has been accepted as an ISO/IEC JTC 1 PAS (“Publicly Available Specification”) Submitter. The OpenChain Specification is the first specification submitted for JTC 1 review<https://www.linuxfoundation.org/press-release/2020/05/joint-development-fou…> and recognition as an international standard. The JDF was formed to simplify the process of creating new technical specification collaboration efforts. Standards and specifications are vitally important for the creation or advancement of new technologies, ensuring that the resulting products are well defined, provide predictable performance and that different implementations can interoperate with one another. […] Announcement link<https://www.linuxfoundation.org/blog/blog/joint-development-foundation-reco…>

1 0

[ANNOUNCE] Apache Daffodil 3.4.0 Released
by Mike Beckerle 08 Nov '22

08 Nov '22

The Apache Daffodil community is pleased to announce the release of version 3.4.0. Notable changes in this release include EXI binary XML support, pluggable character sets, embedded XML, and C code generator updates Detailed release notes and downloads are available at: https://daffodil.apache.org/releases/3.4.0/ Apache Daffodil is an open-source implementation of the DFDL specification that uses DFDL data descriptions to parse fixed format data into an infoset. This infoset is commonly converted into XML or JSON to enable the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of serializing or "unparsing" data back to the original data format. The DFDL infoset can also be converted directly to/from the data structures carried by data processing frameworks so as to bypass any XML/JSON overheads. For more information about Daffodil visit: https://daffodil.apache.org/ Regards, The Apache Daffodil Team

1 0

Fwd: Daffodil Infoset File Size Comparison between xml, exi (schema un-aware) and exisa (schema aware) Parsing
by Mike Beckerle 04 Nov '22

04 Nov '22

I thought this might be of interest to the DFDL WG. ---------- Forwarded message --------- From: Thompson, Dave <dthompson@...> Date: Wed, Oct 19, 2022 at 8:41 AM Subject: Daffodil Infoset File Size Comparison between xml, exi (schema un-aware) and exisa (schema aware) Parsing To: users@... Just a heads up to daffodil users for the upcoming Daffodil v3.4.0 release, which will include EXI support. The attached spreadsheet shows the comparison of Daffodil infoset file size when parsing to xml, exi (schema un-aware) and exisa (schema aware) for some of the Daffodil supported schemas. The comparison shows a significant decrease in infoset size between xml and exi (schema un-aware) infoset files and even a greater reduction with exisa (schema aware) infoset files. In most cases of the exisa (schema aware) infoset files, the infoset file is only slightly larger than the original source file. *Source Data* *Daffodil Infoset File Sizes (bytes)* *Data Format* *Source File (bytes)* *XML* *EXI **(Schema Un-aware)* *EXISA **(Schema aware)* 1 bmp 4,264,316 8,529,476 8,528,926 4,264,305 21,600,054 43,200,952 43,200,406 21,600,044 2 cef 4,180 17,880 2,664 2,462 3 gif 950,734 2,353,949 1,916,689 966,667 4 iCal 5,370 47,345 4,606 4,019 5 Jpeg 1,030,473 2,076,278 2,062,048 1,030,817 15,477 47,023 32,181 15,821 6 Jpeg2000 55,399,819 124,654,407 112,527,466 56,438,548 16,298 41,260 33,989 16,590 7 nato-stanag-5516 (FOUO, not public) 1,952 123,818 5,208 2,366 62,464 3,954,922 122,760 71,682 8 Pcap 10,200 35,361 19,363 10,097 104,858,829 225,950,064 208,409,250 104,806,410 9 png 102,400 206,207 204,809 102,340 10 shp 25,048 255,124 42,122 22,505 2,487,024 14,953,554 3,734,330 3,031,262 11 vmf (FOUO, not public) 1,141 39,744 4,840 1,403

1 0

Fwd: [OpenGridForum/DFDL] WIP: Binary num rep (PR #35)
by Mike Beckerle 04 Nov '22

04 Nov '22

I made a asciidoc that is intended to be reviewable on github, but also produce OGF-style (roughly) PDF output. It's a stub for an experimental features document describing new binary number rep and binary calendar reps. Take a look and let me know what you think. https://github.com/OpenGridForum/DFDL/pull/35

1 0

Agenda for OGF DFDL WG Call
by Steve Hanson 07 Sep '22

07 Sep '22

Please find agenda for next call on GitHub at https://github.com/OpenGridForum/DFDL/blob/master/calls/2022-04-07_DFDL-WG-… Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL<http://www.ibm.com/developerworks/library/se-dfdl/index.html> Co-Chair, OGF DFDL Working Group<http://www.ogf.org/dfdl/> smh(a)uk.ibm.com<mailto:smh@uk.ibm.com> mob:+44-7717-378890 Note: I work Tuesday to Friday

1 8

[Announcement] Apache Daffodil VSCode Extension version 1.1.0
by Mike Beckerle 31 Aug '22

31 Aug '22

The Apache Daffodil community is pleased to announce the release of version 1.1.0 of the Daffodil VS Code Extension. This is the second official release of the Apache Daffodil VS Code Extension. Detailed release notes and downloads are available at: https://daffodil.apache.org/vscode/1.1.0/ The Apache Daffodil VS Code Extension is an open-source extension that utilizes Apache Daffodil to allow users to interactively debug a DFDL schema inside of VS Code. For more information about Daffodil visit: https://daffodil.apache.org/ Regards, The Apache Daffodil Team

1 0

Latest OGF DFDL WG Call Minutes
by Steve Hanson 12 Aug '22

12 Aug '22

Please find minutes from the latest call at https://github.com/OpenGridForum/DFDL/wiki/DFDL-Workgroup-News. Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL<http://www.ibm.com/developerworks/library/se-dfdl/index.html> Co-Chair, OGF DFDL Working Group<http://www.ogf.org/dfdl/> smh(a)uk.ibm.com<mailto:smh@uk.ibm.com> mob:+44-7717-378890 Note: I work Tuesday to Friday

1 3