- dfdl-wg - lists.ogf.org

OGF DFDL WG Call Minutes 2016-01-05
by Steve Hanson 06 Jan '16

06 Jan '16

Please find minutes from the above call at https://redmine.ogf.org/dmsf_files/13511?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Agenda 2016-01-05
by Steve Hanson 04 Jan '16

04 Jan '16

Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13511?download= Please note new time 3pm GMT / 10am EST. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Invitation: OGF DFDL Working Group call (5 Jan 15:00 GMT in IBM: Hursley DE3S09)
by Steve Hanson 16 Dec '15

16 Dec '15

Calls for 2016. OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg DFDL Getting Started: http://www.ibm.com/developerworks/library/se-dfdl/ DFDL 1.0 Spec: http://www.ogf.org/documents/GFD.207.pdf

1 0

OGF DFDL WG Call 2015-12-15 - cancelled
by Steve Hanson 15 Dec '15

15 Dec '15

Steve not available so cancelling. Next meeting is January 5th. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Cancelled: DFDL Working Group Call
by Steve Hanson 15 Dec '15

15 Dec '15

Steve not available

1 0

Cancelled: DFDL Working Group Call
by Steve Hanson 30 Nov '15

30 Nov '15

Mike not available, and no progress on open actions

1 0

Cancelled: DFDL Working Group Call
by Steve Hanson 16 Nov '15

16 Nov '15

Mike unavailable

1 0

Re: [DFDL-WG] EDIFACT schema - daffodil bug or non-bug
by Mike Beckerle 06 Nov '15

06 Nov '15

Bug taken: https://opensource.ncsa.illinois.edu/jira/browse/DFDL-1443 Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> On Fri, Nov 6, 2015 at 1:00 PM, Steve Hanson <smh(a)uk.ibm.com> wrote: > Mike > > I think that is a bug in Daffodil. The DFDL spec says that escapeSchemeRef > applies to simple types with text representation, so Daffodil is evaluating > the escape scheme properties prematurely. If you notice in the UNA > declaration, the simple elements at the start of the UNA that carry the > delimiters and escape character all have dfdl:escapeSchemeRef="" to avoid > tripping the check. > > Regards > > Steve Hanson > Architect, *IBM DFDL* > <http://www.ibm.com/developerworks/library/se-dfdl/index.html> > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > *IBM Integration Bus* > <http://www-03.ibm.com/software/products/en/ibm-integration-bus>, > Hursley, UK > *smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:+44-1962-815848 > mob:+44-7717-378890 > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> > To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> > Date: 04/11/2015 17:17 > Subject: [DFDL-WG] EDIFACT schema - daffodil bug or non-bug > Sent by: "dfdl-wg" <dfdl-wg-bounces(a)ogf.org> > ------------------------------ > > > > > I'm trying to get EDIFACT working on Daffodil. > > I have a somewhat interesting chicken-egg problem. > > This schema uses dfdl:escapeCharacter and dfdl:escapeEscapeCharacter as > expressions. E.g., there is a top-level dfdl:defineVariable named > "EscapeChar" which has a default value, and the expression for the > dfdl:escapeCharacter property is { $ibmEdiFmt:EscapeChar }. > > The default format that is in effect for the root element has > dfdl:lengthKind='delimited'. > > When daffodil starts parsing the top level root/document element, it > enters a parser that is for delimited elements with an escape-scheme in > effect. First thing this parser does is get the escape scheme which > evaluates the expressions for escapeCharacter and escapeEscapeCharacter. > This picks up the default values for those variables and the variables are > then set as "already evaluated", as DFDL specifies that once a variable's > default value has been used, it cannot be subsequently set via > dfdl:setVariable. > > Now, when the very first UNA is encountered, that reads the various > delimiters/escapes from the data, and tries to set the variables. > > But the variables have already been evaluated, on the way into parsing the > "delimited" top level element, and the UNA element itself similarly. > > So it fails with a runtime SDE - default value has already been used. > > So the questions: > > Is this a schema bug in the EDIFACT schema, or is there a principle at > work here indicating that Daffodil cannot evaluate the escape scheme on > entry to an element of length kind delimited unless delimiters are actually > defined? > > It gets worse though. How late bound does this have to be? I can imagine > it being so late as to be after the last child element/group has been > parsed, when the parser unwinds the stack back up to the complex-type > element's tier, and only at that point, when it scans for the terminating > markup, would it then force the evaluation of the escape scheme. But that > seems difficult to implement. However, that would allow the delimiter for > the complex-type element to actually be stored within the children of that > same complex type element. But is this needed? > > One could argue that the EDIFACT schema should have > dfdl:lengthKind='implicit' on these global elements down until the UNA has > been parsed. Though I think that makes authoring schemas harder because a > user thinks of edifact stuff as "a delimited format", and is naturally just > going to want to stick dfdl:lengthKind="delimited" at global scope for all > the schema components. > > Thoughts? > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | > *www.tresys.com* <http://www.tresys.com/> > Please note: Contributions to the DFDL Workgroup's email discussions are > subject to the *OGF Intellectual Property Policy* > <http://www.ogf.org/About/abt_policies.php> > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > https://www.ogf.org/mailman/listinfo/dfdl-wg > >

1 0

Re: [DFDL-WG] minor spec issue - Section 2: required, optional, and RFC2119 notational conventions
by Steve Hanson 06 Nov '15

06 Nov '15

Action raised. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM Integration Bus, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 26/10/2015 11:09 Subject: Re: [DFDL-WG] minor spec issue - Section 2: required, optional, and RFC2119 notational conventions Mike, I agree with your conclusion. In RFC2119 'must' and 'shall' are also synonyms, and I would like to see the DFDL spec use 'must' throughout and drop the use of 'shall' (it looks like there is only one use - in section 9.3.2.2). Fwiw I've always been suspicious of that statement in section 2, as I don't believe there has ever been a review of the uses of 'must', 'should', 'may' to ensure compliance with RFC2119. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM Systems, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 25/10/2015 21:55 Subject: [DFDL-WG] minor spec issue - Section 2: required, optional, and RFC2119 notational conventions Sent by: "dfdl-wg" <dfdl-wg-bounces(a)ogf.org> In Section 2 on notational conventions we say "The key words must, must not, required, shall, shall not, should, should not, recommended, may, may not and optional in this document are to be interpreted as described in [RFC2119]." Which is fine except for the words "required" and "optional" which we use in several different senses. Section 21 on optional features of the DFDL standard uses "optional" vs. "required" in this sense of RFC2119. But we also use "Optional Occurrence", "Optional Element" very specifically and define them in our glossary. (Along with Required Occurrence and Required Element.) So the above sentence on notational conventions we should just drop the words "optional" and "required". I looked for synonyms for required/optional. mandatory/nonmandatory and compulsory/noncompulsory are ones that we might consider using in the future. We do use mandatory now as in mandatory alignment of character set code units. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg

1 0

EDIFACT schema - daffodil bug or non-bug
by Mike Beckerle 06 Nov '15

06 Nov '15

I'm trying to get EDIFACT working on Daffodil. I have a somewhat interesting chicken-egg problem. This schema uses dfdl:escapeCharacter and dfdl:escapeEscapeCharacter as expressions. E.g., there is a top-level dfdl:defineVariable named "EscapeChar" which has a default value, and the expression for the dfdl:escapeCharacter property is { $ibmEdiFmt:EscapeChar }. The default format that is in effect for the root element has dfdl:lengthKind='delimited'. When daffodil starts parsing the top level root/document element, it enters a parser that is for delimited elements with an escape-scheme in effect. First thing this parser does is get the escape scheme which evaluates the expressions for escapeCharacter and escapeEscapeCharacter. This picks up the default values for those variables and the variables are then set as "already evaluated", as DFDL specifies that once a variable's default value has been used, it cannot be subsequently set via dfdl:setVariable. Now, when the very first UNA is encountered, that reads the various delimiters/escapes from the data, and tries to set the variables. But the variables have already been evaluated, on the way into parsing the "delimited" top level element, and the UNA element itself similarly. So it fails with a runtime SDE - default value has already been used. So the questions: Is this a schema bug in the EDIFACT schema, or is there a principle at work here indicating that Daffodil cannot evaluate the escape scheme on entry to an element of length kind delimited unless delimiters are actually defined? It gets worse though. How late bound does this have to be? I can imagine it being so late as to be after the last child element/group has been parsed, when the parser unwinds the stack back up to the complex-type element's tier, and only at that point, when it scans for the terminating markup, would it then force the evaluation of the escape scheme. But that seems difficult to implement. However, that would allow the delimiter for the complex-type element to actually be stored within the children of that same complex type element. But is this needed? One could argue that the EDIFACT schema should have dfdl:lengthKind='implicit' on these global elements down until the UNA has been parsed. Though I think that makes authoring schemas harder because a user thinks of edifact stuff as "a delimited format", and is naturally just going to want to stick dfdl:lengthKind="delimited" at global scope for all the schema components. Thoughts? Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1