July 2019 - dfdl-wg - lists.ogf.org

clarification needed: unparsing and hidden group choices and choices with dfdl:choiceDispatchKey
by Mike Beckerle 13 Dec '19

13 Dec '19

So interesting issue. Comes up as part of our experience implementing the advanced expression language features: hidden groups, and computed elements. We have hidden groups in DFDL. If a xs:choice is inside a hidden group, then by definition, when unparsing, elements corresponding to some branch of the choice are not in the Infoset. DFDL spec clearly states that dfdl:outputValueCalc is not allowed on the head of a choice branch. But elements inside hidden groups must have dfdl:outputValueCalc, or a default value, otherwise there's no way for them to be filled in as part of the augmented infoset. So the only way a choice can be inside a hidden group is if the head of the choice branches have default values. This is not sufficient. I do not think it was our intention to make hidden groups incapable of holding things complex enough to contain a choice. Rather, the point is to be able to hide things that have excess complexity needed only for parsing/unparsing, so that they need not appear in the final Infoset. So if a choice is allowed inside a hidden group, then a mechanism is needed by which when unparsing a choice branch is selected based on values of other parts of the Infoset, and then the elements of that branch are computed and added to the augmented infoset. Note that without hidden groups, this problem never arises, as the Infoset always has to take a position on choice branches by having an unambiguous element in it. There are 3 cases: 1. unparsing an xs:choice where for parsing, each branch has a discriminator, or is resolved by the simple "first successful parse" rule. 2. unparsing an xs:choice with dfdl:choiceDispatchKey 3. unparsing xs:choice with dfdl:initiatedContent='yes' For all cases, I suggest we must remove the restriction on dfdl:outputValueCalc being disallowed on choice branch head. For (1). First branch where the branch can be constructed successfully is added to the augmented infoset. This is a case where an unparse error is non-fatal, and causes a form of unparser backtracking through alternatives. Branches would be attempted in schema order. For (2) I suggest dfdl:unparseChoiceDispatchKey, evaluated at unparse time only. Produces a choice branch key. That choice branch is then constructed, and default values or dfdl:outputValueCalcs are used to fill in values. (BTW: this is the driving use case we have in Daffodil for this issue.) For (3).... I have no specific solution. I think it may not actually be a separate case and has to be treated as (1) or (2). Comments/Discussion?

2 1

BLOB - binary large object proposal - updated
by Mike Beckerle 29 Aug '19

29 Aug '19

This concept, ,which has been discussed before, is in high demand in the Daffodil user community to enable DFDL to be used to parse image file formats. The use case is to provide uniform image-metadata access without getting bogged down in the large byte-array that makes up most of the file and would be very large (and pointless) if rendered into XML or JSON. So our proposal, (which will get turned into an official Experimental feature document), has been simplified and revised and is described here: https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Binary+Lar… Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

3 5

Latest OGF DFDL WG Call Minutes
by Steve Hanson 11 Jul '19

11 Jul '19

Please find minutes from the latest call at https://redmine.ogf.org/projects/dfdl-wg/news Regards Steve Hanson IBM Hybrid Integration Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890

1 0

Action 228 - new DFDL Tutorials published
by Mike Beckerle 11 Jul '19

11 Jul '19

Roger Costello of MITRE maintains an excellent site on all things about XML called xFront. He has just released a bunch of new DFDL tutorial materials including slides, lab exercises, etc. at http://www.xfront.com/DFDL/index.html These were developed using the Apache Daffodil (incubating) DFDL implementation. They have not been reviewed or tested for portability to other DFDL implementations. There are slides that use DFDL features that are not yet implemented by other DFDL implementations, but much of the material should be about the core portable set of concepts and properties. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

OGF DFDL Working Group call
by Steve Hanson 11 Jul '19

11 Jul '19

Calls for 2019. OGF DFDL WG weekly call dial-in details. https://ibm.webex.com/meet/smh OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg DFDL Getting Started: http://www.ibm.com/developerworks/library/se-dfdl/ DFDL 1.0 Spec: http://www.ogf.org/documents/GFD.207.pdf Daffodil Apache incubator: https://daffodil.apache.org/ IBM DFDL: https://www.ibm.com/support/knowledgecenter/en/SSTTDS_11.0.0/com.ibm.etools…

1 0

Confirmed: OGF DFDL Working Group call (8 Aug 16:00 GDT)
by Steve Hanson 11 Jul '19

11 Jul '19

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Information Update - Description has changed: OGF DFDL Working Group call
by Steve Hanson 11 Jul '19

11 Jul '19

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Information Update - Description has changed: OGF DFDL Working Group call
by Steve Hanson 11 Jul '19

11 Jul '19

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Confirmed: OGF DFDL Working Group call (11 Jul 16:00 GDT)
by Steve Hanson 11 Jul '19

11 Jul '19

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Agenda for OGF DFDL WG Call
by Steve Hanson 11 Jul '19

11 Jul '19

Please find agenda for today's call on Redmine at https://redmine.ogf.org/dmsf_files/13591?download= I have an X-ray on my knee at my local hospital today, it should hopefully be over by the call. If I am delayed I will send a further email. Regards Steve Hanson IBM Hybrid Integration Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890

1 0