- dfdl-wg - lists.ogf.org

OGF DFDL WG Call Agenda 2014-11-11
by Steve Hanson 10 Nov '14

10 Nov '14

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13373?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call 2014-11-04 - cancelled
by Steve Hanson 04 Nov '14

04 Nov '14

Today's call is cancelled as not enough people available. Next meeting is 11th. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Cancelled: DFDL Working Group Call
by Steve Hanson 04 Nov '14

04 Nov '14

Cancelling call as not enough people available

1 0

Re: [DFDL-WG] Trimming of a text number that's all zeros when the number pattern has a sign char at the end
by Andrew Edwards 03 Nov '14

03 Nov '14

Hi Steve Yep - I agree with your new definition of the rule taking into account justification-independence and quoted text. I'm not sure I like the nested if-else in the new text though, as it took a couple of reads to understand. How about the text below? It splits out the second 'if/else', which I find easier to understand. When parsing, if the pad character is '0' and dfdl:textTrimKind is 'padChar' then the SimpleContent region is trimmed of the '0' characters as defined by the trimming rules. If this trimming results in the next character in the SimpleContent region being a character other than a digit, the last '0' character is re-instated and not trimmed. This rule also applies when the pad character is a DFDL character entity equivalent to '0'. This rule does not apply when the pad character is any other character nor when a pad byte is specified. Cheers, Andy Andy Edwards - IBM Integration Bus - DFDL Email: andy.edwards(a)uk.ibm.com Snail Mail: MP211, Hursley park, Hursley, WINCHESTER, Hants, SO21 2JN Tel int: 247222 Tel ext: +44 (0)1962 817222 Desk: DE3 V17 The Feynman problem solving Algorithm 1) Write down the problem 2) Think real hard 3) Write down the answer -- Murray Gell-mann in the NY Times From: Steve Hanson/UK/IBM To: Andrew Edwards/UK/IBM@IBMGB Cc: DFDL-WG <dfdl-wg(a)ogf.org> Date: 03/11/2014 15:39 Subject: Re: [DFDL-WG] Trimming of a text number that's all zeros when the number pattern has a sign char at the end Andy I agree that the existing words do not cover all the scenarios. Your proposed words are on the right track but only cover left trimming a right justified text number. We need something that is independent of justification and can handle patterns where there is quoted text as well as signs. One can envisage some bizarre scenarios. Eg, Text number pattern is "#0'000'" - an attempt to divide by 1000 using the pattern. DFDL parser would trim everything except 1 zero which would not match the pattern which expects at least 3 zeros. Trimming happens before pattern is looked at so I don't think we could cater for this (if we even wanted to). Perhaps we should say: When parsing, if the pad character is '0' and dfdl:textTrimKind is 'padChar' then if the SimpleContent region is trimmed so that the removal of a '0' character leaves the next character other than a digit, the last '0' character is re-instated and not trimmed. This rule also applies when the pad character is a DFDL character entity equivalent to '0'. This rule does not apply when the pad character is any other character nor when a pad byte is specified. That means that "000,000,123" would end up as "0,000,123" instead of ",000,123" today and "0000.025" would end up as "0.025" instead of ".025" today but I think that is good. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Andrew Edwards/UK/IBM@IBMGB To: DFDL-WG <dfdl-wg(a)ogf.org> Date: 30/10/2014 16:43 Subject: [DFDL-WG] Trimming of a text number that's all zeros when the number pattern has a sign char at the end Sent by: dfdl-wg-bounces(a)ogf.org Hi all, I've hit an interesting case revolving around trimming and number patterns that doesn't seem quite sane to me. Consider an element with the following properties: textTrimKind='padChar' textNumberPadCharacter='0' textNumberPattern='0000+;0000-' So we have the sign character at the end of the representation. Now, imagine that the data being parsed is "0000+". The relevant rules from the DFDL specification are: Section 13.2 on textTrimKind When 'padChar', the element is trimmed of the dfdl:textStringPadCharacter, dfdl:textNumberPadCharacter, dfdl:textBooleanPadCharacter or dfdl:textCalendarPadCharacter depending on the type of the element. Section 13.6 on textNumberPadCharacter When parsing, if the pad character is '0' and the SimpleContent region consists entirely of '0' characters, then the last remaining '0' is not trimmed and a single '0' is the result of the trimming. This rule also applies when the pad character is a DFDL character entity equivalent to '0'. This rule does not apply when the pad character is any other character nor when a pad byte is specified. Section 13.6.1 Describes all of the pattern syntax. In our hypothetical case, the content region is not all zeros, as it ends in '+'. This means that the rule in section 13.6 does not apply and we only apply the rule in 13.2. This results in us trimming away all of the zeros and ending up with '+'. This then doesn't parse as a number. The problem seems to be that the rule in Section 13.6 doesn't take into account that the suffix of the pattern can result in text in the content region that isn't part of the digits of the number. Should the rule under section 13.56 be something more like this... When parsing, if the pad character is '0' and the SimpleContent region consists entirely of '0' characters, or the SimleContent region consists of a string of '0' characters followed by non-digit characters, then the last remaining '0' is not trimmed and a single '0' is the result of the trimming. This rule also applies when the pad character is a DFDL character entity equivalent to '0'. This rule does not apply when the pad character is any other character nor when a pad byte is specified. Thoughts? Andy Andy Edwards - IBM Integration Bus - DFDL Email: andy.edwards(a)uk.ibm.com Snail Mail: MP211, Hursley park, Hursley, WINCHESTER, Hants, SO21 2JN Tel int: 247222 Tel ext: +44 (0)1962 817222 Desk: DE3 V17 The Feynman problem solving Algorithm 1) Write down the problem 2) Think real hard 3) Write down the answer -- Murray Gell-mann in the NY Times Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Trimming of a text number that's all zeros when the number pattern has a sign char at the end
by Andrew Edwards 03 Nov '14

03 Nov '14

Hi all, I've hit an interesting case revolving around trimming and number patterns that doesn't seem quite sane to me. Consider an element with the following properties: textTrimKind='padChar' textNumberPadCharacter='0' textNumberPattern='0000+;0000-' So we have the sign character at the end of the representation. Now, imagine that the data being parsed is "0000+". The relevant rules from the DFDL specification are: Section 13.2 on textTrimKind When 'padChar', the element is trimmed of the dfdl:textStringPadCharacter, dfdl:textNumberPadCharacter, dfdl:textBooleanPadCharacter or dfdl:textCalendarPadCharacter depending on the type of the element. Section 13.6 on textNumberPadCharacter When parsing, if the pad character is '0' and the SimpleContent region consists entirely of '0' characters, then the last remaining '0' is not trimmed and a single '0' is the result of the trimming. This rule also applies when the pad character is a DFDL character entity equivalent to '0'. This rule does not apply when the pad character is any other character nor when a pad byte is specified. Section 13.6.1 Describes all of the pattern syntax. In our hypothetical case, the content region is not all zeros, as it ends in '+'. This means that the rule in section 13.6 does not apply and we only apply the rule in 13.2. This results in us trimming away all of the zeros and ending up with '+'. This then doesn't parse as a number. The problem seems to be that the rule in Section 13.6 doesn't take into account that the suffix of the pattern can result in text in the content region that isn't part of the digits of the number. Should the rule under section 13.56 be something more like this... When parsing, if the pad character is '0' and the SimpleContent region consists entirely of '0' characters, or the SimleContent region consists of a string of '0' characters followed by non-digit characters, then the last remaining '0' is not trimmed and a single '0' is the result of the trimming. This rule also applies when the pad character is a DFDL character entity equivalent to '0'. This rule does not apply when the pad character is any other character nor when a pad byte is specified. Thoughts? Andy Andy Edwards - IBM Integration Bus - DFDL Email: andy.edwards(a)uk.ibm.com Snail Mail: MP211, Hursley park, Hursley, WINCHESTER, Hants, SO21 2JN Tel int: 247222 Tel ext: +44 (0)1962 817222 Desk: DE3 V17 The Feynman problem solving Algorithm 1) Write down the problem 2) Think real hard 3) Write down the answer -- Murray Gell-mann in the NY Times Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

OGF DFDL WG Call Agenda 2014-11-04
by Steve Hanson 03 Nov '14

03 Nov '14

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13370?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Minutes 2014-10-28
by Steve Hanson 03 Nov '14

03 Nov '14

Please find minutes from the above call at http://redmine.ogf.org/dmsf_files/13369?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

elementFormDefault and the form attribute on element declarations
by Mike Beckerle 03 Nov '14

03 Nov '14

Neither the term "elementFormDefault" nor the term "unqualified" appear in the DFDL Spec. So we're silent about whether these are allowed or not in DFDL schemas. We should specify what of this XML Schema behavior is allowed, which I believe means add a line to each of the lists in section 5.1 of things used and not-used/reserved from XML Schema. FYI: Today the Daffodil implementation implements elementFormDefault on the xs:schema but not the form attribute on element declarations. We have tests that exercise elementFormDefault in qualified and unqualified and not specified at all (which defaults to "unqualified" behavior in XSD, so we're consistent with that.) Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

Action 274 - on qnames in path expressions in DFDL
by Mike Beckerle 03 Nov '14

03 Nov '14

I did some experiments with the XML validation built into eclipse, and the key constraint, which has paths (in the selector and field attributes) that refer to other parts of the XML infoset/document and so involve the names of elements. I also have been reading the XPath 2.0 specs including the formal semantics. To my interpretation XPath 2.0 is pretty explicit: An element or type QName consisting only of a local part NCName expands to the default element/type namespace and the local part. They don't qualify this in any way that I can find for the names used in path steps. The formal semantics is pretty hard to understand, but this detail seems to be swept under the rug. There would need to be specific treatment of this in the naming environment, but I cannot find this. There is discussion of name resolution in some of their judgements, but nothing I can see that creates an exception for names in path steps. (Called name tests) What I was looking for was language that specifies something about a non-prefixed name in a path step and that it is NOT augmented by the default namespace. I could find no such exception. But if I am right and this detail isn't there, i.e., If there is no such exception, then the implementation of XPath in eclipse (which is xerces I believe) is incorrect in the way it resolves names that appear in path steps. Here's how it works according to eclipse: In xpath expressions in the selectors and fields of unique and key constraints, xmlns (the default namespace) is not considered as implicitly augmenting unqualified names that appear in path steps. An identifier in a path step that has no prefix (e.g., 'foo' in the path './p1:bar/foo/p2:baz') is always interpreted as identifying either 1) a child element defined by a local element declaration with form "unqualified". or 2) this corner case: a child element defined by an element reference to a global element declaration but only when that global element declaration has no effective target namespace. (Use of 'effective' here rules out an element decl inside a schema document that itself has no target namespace but is included into a schema that has one. Such an element declaration effectively has a target namespace, so this corner case doesn't apply.) Case (2) here is a rather obscure corner case, but there nevertheless. In no case (that I could construct) does the name in a path step get augmented by a schema's default namespace (defined by xmlns="...whatever..."). The above is true for names without prefix appearing in path steps of expressions. The QNames that are used in an XML schema to reference a global declaration or definition (in the ref of an element ref, type attribute in a type reference, base in a simple type derivation, etc.) are different. Those names DO pickup a default namespace if one is defined and they have no explicit prefix themselves. This creates a corner case for these references (to global decls/defs) which is if a definition/declaration appears in no effective target namespace, then there is no way to reference it at all from a schema that has a default namespace definition surrounding the point of reference. There is no syntactic way to turn-off use of a default namespace. (The semantics of xpath allows for a "none" default namespace definition, but there is no xml syntax for expressing explicitly this "none". It is simply what you have if there is no prefix and no default namespace). You can change which namespace the default is, but there's no way to say explicitly "I meant this QName reference to be to something not in any namespace. Unless/until we can find language in some XML Schema spec indicating how path step names (name tests) are specifically resolved, then the above is just "how one system does it." But we may end up having to follow precedent here from implementations. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

3 4

clarification: fn:exactly-one function
by Mike Beckerle 03 Nov '14

03 Nov '14

According to the DFDL spec, fn:exactly-one returns "True if the provided sequence contains exactly one node/value," which suggests that the function would return false if the sequence contains > or < 1 node/value. The XPath description states that fn:exactly-one "returns $arg if it contains exactly one item. Otherwise, raises an error." It seems of value to have a true/false result, because something that raises an error like this can only be used as an assertion effectively. On the other hand, that's not what XPath does. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

3 2