- dfdl-wg - lists.ogf.org

Re: [DFDL-WG] Action 272: Catch 22.for EVDP & NVDP ?
by Steve Hanson 18 Nov '14

18 Nov '14

Review for next call please. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, DFDL-WG <dfdl-wg(a)ogf.org> Date: 28/10/2014 14:48 Subject: Re: [DFDL-WG] Catch 22.for EVDP & NVDP ? Thinking through the implications of having EVDP not apply if content length can not be zero: 1) Cases are dfdl:lengthKind 'explicit', dfdl:length not an expression and not zero, and (simple only) dfdl:lengthKind 'implicit' and maxLength facet not zero. 2) Does it mean a) that there is no concept of an empty representation, or b) that there is but it is not subject to EVDP and 'effective' EVDP is 'none' ? - If b) that does not fix the issue around initiatedContent, - if a) it means not finding the element must be taken as missing which implies no default value can be applied. Hmm 3) What about NVDP ? Should we be disallowing the use of %ES; as a nil value for these cases as well ? Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 17/09/2014 15:44 Subject: Re: [DFDL-WG] Catch 22.for EVDP & NVDP ? Having thought this through I think that you are correct. If the content length can never be 0 then EVDP does not apply. I think IBM DFDL is issuing an error when it should not be. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 09/09/2014 15:37 Subject: Re: [DFDL-WG] Catch 22.for EVDP & NVDP ? Interesting. To me if lengthKind is explicit, and non-variable (i.e., is fixed length), then EVDP doesn't apply, because by definition, such a representation is not ever empty. I would assume that EVDP would be ignored in such a case, and an implementation might even generate warnings if dfdl:emptyValueDelimiterPolicy appeared directly on such an element or simpleType where a fixed length is also expressed. This is one of the challenges that having lengthKind 'explicit' which handles both the fixed length, and variable-length using an expression, creates. Even though we don't have a lengthKind 'fixed', we still have fixed-length as a concept, and some other features interact with fixed length, like empty values. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Tue, Sep 2, 2014 at 10:31 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: <xs:element name="choiceGroup" dfdl:lengthKind="implicit"> <xs:complexType> <xs:choice dfdl:choiceLengthKind="implicit" dfdl:initiatedContent="yes"> <xs:element name="myNum1" type="xs:int" dfdl:lengthKind="explicit" dfdl:length="2" dfdl:textNumberPattern="#0" dfdl:initiator="num:" dfdl:terminator="~" /> > <xs:element name="myText1" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="2" dfdl:initiator="text:" dfdl:terminator="~" /> </xs:choice> </xs:complexType> </xs:element> The above model is a choice of two elements, each of which is fixed length and has an initiator and terminator. Because the elements are fixed length, I must set dfdl:emptyValueDelimiterPolicy (EVDP) to be 'none' **. I also want to discriminate using the initiator, so I set dfdl:initiatedContent 'yes' on the choice, so I must set EVDP to 'initiator' or 'both' ++. (Same for NVDP if nillable). Catch 22. ** 'none' because the only valid empty representation is that there is nothing there at all. The element is fixed length X, so if the delimiters are present then the content must be length X and can't be zero length. ++ 'initiator' or 'both' because an initiator must always be present when dfdl:initiatedContent 'yes'. The implication is that I can't use dfdl:initiatedContent for this choice. While I can switch off nillable to avoid the error with NVDP, there is no other way of avoiding the EVDP error. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

Re: [DFDL-WG] Action 274 - on qnames in path expressions in DFDL
by Tim Kimber 18 Nov '14

18 Nov '14

You're right - I did not realize that this 'no undeclaring of a non-empty prefix' rule existed. http://www.w3.org/TR/REC-xml-names/#nsc-NoPrefixUndecl So this is not valid: <foo:root xmlns:foo="www.tk.com"> <foo:child xmlns:foo=""/> </foo:root> but this is valid: <root xmlns="www.tk.com"> <child xmlns=""/> </root> So there is a way to change the default namespace from "someURL" to "". And it should therefore be possible to re-assign the default namespace in an element reference so that it can refer to a global element declared in a noTargetNamespace xsd. regards, Tim Kimber, Technical Lead for IBM Integration Bus Healthcare Pack Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Steve Hanson/UK/IBM To: Tim Kimber/UK/IBM@IBMGB Cc: dfdl-wg(a)ogf.org Date: 03/11/2014 14:38 Subject: Re: [DFDL-WG] Action 274 - on qnames in path expressions in DFDL Tim, I'm not sure that you comparing apples with apples here. Mike is asking whether xmlns:foo="" is valid, whereas your stackoverflow link is talking about xmlns="" and it's specific use in an XML document to undeclare the namespace in scope. Are you sure that xmlns:foo="" is valid ? Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org Date: 17/10/2014 03:55 Subject: Re: [DFDL-WG] Action 274 - on qnames in path expressions in DFDL Sent by: dfdl-wg-bounces(a)ogf.org I think Eclipse must be doing something non-standard then: http://stackoverflow.com/questions/1587891/is-xmlns-a-valid-xml-namespace Which version of Eclipse / Eclipse plugin are you using? regards, Tim Kimber, Technical Lead for IBM Integration Bus Healthcare Pack Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Tim Kimber/UK/IBM@IBMGB Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 16/10/2014 14:52 Subject: Re: [DFDL-WG] Action 274 - on qnames in path expressions in DFDL Tim, How do you do this? => "You define a non-empty prefix that points to an empty namespace, and use that prefix in the QName." specifically, what XML text corresponds to "an empty namespace". xmlns:foo="" isn't allowed (in eclipse anyway) Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Thu, Oct 16, 2014 at 12:58 AM, Tim Kimber <KIMBERT(a)uk.ibm.com> wrote: My off-the-cuff reactions: - every conforming XPath processor allows a 'namespace context' to be defined. The context defines a prefix->namespace URI mapping for zero or more prefixes. - namespace prefixes ( including the empty prefix ) are then interpreted in exactly the same way as they would be in an XML document. This makes sense, because XPaths were explicitly designed to be included within XML documents. It would look rather strange if the prefixes within an XPath were to be interpreted differently by a) an XML processor and b) an XPath processor. I think that answers most of the queries, although it raises a question over whether the Eclipse implementation is conforming. re: the question about empty prefixes, there is a (non-obvious) way to refer to a noTargetNamespace global element from within a document that has a default namespace. You define a non-empty prefix that points to an empty namespace, and use that prefix in the QName. regards, Tim Kimber, Technical Lead for IBM Integration Bus Healthcare Pack Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 16/10/2014 01:41 Subject: [DFDL-WG] Action 274 - on qnames in path expressions in DFDL Sent by: dfdl-wg-bounces(a)ogf.org I did some experiments with the XML validation built into eclipse, and the key constraint, which has paths (in the selector and field attributes) that refer to other parts of the XML infoset/document and so involve the names of elements. I also have been reading the XPath 2.0 specs including the formal semantics. To my interpretation XPath 2.0 is pretty explicit: An element or type QName consisting only of a local part NCName expands to the default element/type namespace and the local part. They don't qualify this in any way that I can find for the names used in path steps. The formal semantics is pretty hard to understand, but this detail seems to be swept under the rug. There would need to be specific treatment of this in the naming environment, but I cannot find this. There is discussion of name resolution in some of their judgements, but nothing I can see that creates an exception for names in path steps. (Called name tests) What I was looking for was language that specifies something about a non-prefixed name in a path step and that it is NOT augmented by the default namespace. I could find no such exception. But if I am right and this detail isn't there, i.e., If there is no such exception, then the implementation of XPath in eclipse (which is xerces I believe) is incorrect in the way it resolves names that appear in path steps. Here's how it works according to eclipse: In xpath expressions in the selectors and fields of unique and key constraints, xmlns (the default namespace) is not considered as implicitly augmenting unqualified names that appear in path steps. An identifier in a path step that has no prefix (e.g., 'foo' in the path './p1:bar/foo/p2:baz') is always interpreted as identifying either 1) a child element defined by a local element declaration with form "unqualified". or 2) this corner case: a child element defined by an element reference to a global element declaration but only when that global element declaration has no effective target namespace. (Use of 'effective' here rules out an element decl inside a schema document that itself has no target namespace but is included into a schema that has one. Such an element declaration effectively has a target namespace, so this corner case doesn't apply.) Case (2) here is a rather obscure corner case, but there nevertheless. In no case (that I could construct) does the name in a path step get augmented by a schema's default namespace (defined by xmlns="...whatever..."). The above is true for names without prefix appearing in path steps of expressions. The QNames that are used in an XML schema to reference a global declaration or definition (in the ref of an element ref, type attribute in a type reference, base in a simple type derivation, etc.) are different. Those names DO pickup a default namespace if one is defined and they have no explicit prefix themselves. This creates a corner case for these references (to global decls/defs) which is if a definition/declaration appears in no effective target namespace, then there is no way to reference it at all from a schema that has a default namespace definition surrounding the point of reference. There is no syntactic way to turn-off use of a default namespace. (The semantics of xpath allows for a "none" default namespace definition, but there is no xml syntax for expressing explicitly this "none". It is simply what you have if there is no prefix and no default namespace). You can change which namespace the default is, but there's no way to say explicitly "I meant this QName reference to be to something not in any namespace. Unless/until we can find language in some XML Schema spec indicating how path step names (name tests) are specifically resolved, then the above is just "how one system does it." But we may end up having to follow precedent here from implementations. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 2

Fw: Fw: Action 260
by Steve Hanson 18 Nov '14

18 Nov '14

IBM has discussed this issue at some length internally, and come to the conclusion that the optional and array elements in a sequence should follow the separator suppression policy (SSP), and not have an implied SSP which is at odds with the sequence's SSP. In the table below, cells marked with a cross imply a schema definition error, and cells marked ok imply that there is a behaviour of the element for that OCK which is in keeping with the SSP of the sequence. SSP (1) OCK fixed implicit expression parsed stopValue (2) never ok ok ok x (5) ok (6) trailingEmpty trailingEmptyStrict ok (3) ok ok (4) x (5) ok (6) anyEmpty ok (3) ok ok (4) ok ok Notes: (1) SSP property applies only to an ordered sequence. An unordered sequence assumes 'anyEmpty' (as all optional/array elements must be 'parsed') (2) Missing restriction - for 'stopValue' the dfdl:stopValue property must not include empty string. (3) maxOccurs provides count, so nothing is eligible for suppression, so SSP has no practical effect (same as a required element) (4) infoset provides count, so nothing is eligible for suppression, so SSP has no practical effect (same as a required element) (5) 'parsed' only makes sense with 'anyEmpty' (6) Because a stop value must appear, and from (2) empty string is not allowed, SSP has no practical effect. The issue of maxOccurs = '0' is discussed in a separate email. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 13/10/2014 14:40 ----- From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 29/08/2014 21:21 Subject: Re: [DFDL-WG] Fw: Action 260 I reviewed this. It looks good to me. The note at the bottom that we don't say what happens on a zero-trip I.e., a represented element, but where occursCount evaluates to 0, is a useful clarification also. Do we want to create an erratum for this? Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Thu, Aug 28, 2014 at 10:13 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: Please review for Tuesday's WG call ... Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 28/08/2014 15:02 ----- From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org, Date: 06/08/2014 13:50 Subject: Fw: [DFDL-WG] Action 260 So my suggestion below, to wrap the array in a sequence, does not work; it just moves the problem down into the new sequence. After much deliberation, we think that the definitions of Positional sequence and Non-positional sequence should not be viewed as driving the behaviour of a sequence, but simply as the resultant characteristics of a sequence that has certain properties. That leaves modellers free to mix occursCountKinds, as in Tim's example. No need for any new SDE scenarios. Positional sequence - Each occurrence in the sequence can be identified by its position in the data. Typically the components of such a sequence do not have an initiator. In some such sequences, the separators for optional zero-length occurrences may or must be omitted when at the end of the group. In DFDL, a sequence is considered positional if it contains only required elements and/or optional and array elements that have dfdl:occursCountKind 'implicit', 'fixed' or 'expression', and it has dfdl:separatorSuppressionPolicy 'never', 'trailingEmptyStrict' or 'trailingEmpty'. Non-positional sequence - Occurrences in the sequence cannot be identified by their position in the data alone. Often the components of such a sequence have an initiator. Such sequences sometimes allow the separator to be omitted for optional zero-length occurrences anywhere in the sequence. Speculative parsing might need to be employed by to identify each occurrence. In DFDL, a sequence is non-positional if it contains any optional or array elements that have dfdl:occursCountKind 'parsed' or 'stopValue', and/or it has dfdl:separatorSuppressionPolicy 'anyEmpty'. See parallel email for action 261 that ensures 'expression' behaves itself. One behaviour that is missing from the spec. For a sequence with separators, what is expected in the data stream if occursCount = 'fixed' / 'implicit' and maxOccurs = '0', or occursCountKind = 'expression' and occursCount evaluates to 0 ? We believe that no separator should be expected when parsing and none output when unparsing (same behaviour as inputValueCalc). Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 06/08/2014 12:42 ----- From: Steve Hanson/UK/IBM To: Tim Kimber/UK/IBM@IBMGB, Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Date: 30/06/2014 10:04 Subject: Re: [DFDL-WG] Action 260 You would wrap the array and it's count in a sequence. Then the 'count+array' is treated as a single entity as far as the parent sequence is concerned. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org, Date: 26/06/2014 20:06 Subject: Re: [DFDL-WG] Action 260 Sent by: dfdl-wg-bounces(a)ogf.org Before we settle one way or the other, I would like the following data format to be taken into consideration. chars,5,A,B,C,D,E,integers,1,2,3 chars,3,C,,,integers,2,10,11 I am assuming that the occursCountKind for the arrays is 'expression' and the occursCount refers to the integer field that precedes the array. In order to represent the empty strings on the second line it is essential to specify SSP as 'trailingEmpty' or 'never'. If we disallow the combination of ock='expression' and SSP='trailingEmpty' then how would this format be modelled? regards, Tim Kimber, Technical Lead for IBM Integration Bus Healthcare Pack Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 25/06/2014 16:25 Subject: Re: [DFDL-WG] Action 260 Sent by: dfdl-wg-bounces(a)ogf.org I prefer choice (a) for two reasons * It is more restrictive and therefore more conservative (preserving freedom to change in future if needed) * If a user has a positional data format, you don't want them to even have to understand the concept of speculation in order to model their data. So choice (a) allows a simpler description that doesn't need to introduce the notion that the parser might be speculation. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Wed, Jun 25, 2014 at 5:20 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: 260 Positional and non-positional sequences (All) 10/6: Spec defines the above but also allows different occursCountKinds within the same sequence which may have different (implied) separatorSuppressionPolicy, which results in a sequence which is a mixture of both. Should this be allowed? If so what are the rules? Can certain combinations be disallowed? 17/6: IBM have discussed internally and will submit a proposal. In the spec we define Positional Sequence and Non-Positional Sequence: Positional sequence - Each occurrence in the sequence can be identified by its position in the data. Typically the components of such a sequence do not have an initiator. In some such sequences, the separators for optional zero-length occurrences may or must be omitted when at the end of the group. A positional sequence can be modelled by setting dfdl:separatorSuppressionPolicy to 'never', 'trailingEmptyStrict' or 'trailingEmpty'. Non-positional sequence - Occurrences in the sequence cannot be identified by their position in the data alone. Typically the components of such a sequence have an initiator. Such sequences allow the separator to be omitted for optional zero-length occurrences anywhere in the sequence. Speculative parsing is employed by the parser to identify each occurrence. A non-positional sequence can be modelled by setting dfdl:separatorSuppressionPolicy to 'anyEmpty'. The problem is that the setting of dfdl:separatorSuppressionPolicy is only examined for child elements with dfdl:occursCountKind 'implicit'. For other dfdl:occursCountKinds, there is the concept of an 'implied' dfdl:separatorSuppressionPolicy: When dfdl:occursCountKind is 'fixed' then ... the implied behaviour is 'never'. When dfdl:occursCountKind is 'expression' ... the implied behaviour is 'never'. When dfdl:occursCountKind is 'parsed' ... the implied behaviour is 'anyEmpty'. When dfdl:occursCountKind is 'stopValue' ...the implied behaviour is 'anyEmpty'. So if a Positional sequence as defined above contains children with dfdl:occursCountKind 'parsed' or 'stopValue' then surely it is no longer a Positional sequence. A solution to this is to prevent the appearance of certain values of dfdl:occursCountKind within a Positional sequence. However, precisely which values to outlaw is subject to interpretation of the phrase "Each occurrence in the sequence can be identified by its position in the data". Is this intended to mean: a) an observer of the raw data can identify an occurrence of an element in the sequence solely by counting separators => SDE if 'parsed', 'stopValue' or 'expression' ** appeared in a Positional sequence; ** Although 'expression' would appear to be like 'fixed' it actually breaks a) so must be included in the SDE list. or b) a parser does not have to speculate to identify an occurrence of an element in the sequence => SDE only if 'parsed' appeared in a Positional sequence. Note that it is possible to wrap a 'parsed' etc element in a local sequence or another element to avoid an SDE. But this could still be seen as a violation of a) if the separators of both are the same, as the observer can not count the separators. So should the rule be applied recursively, ie, a Positional sequence can not contain a non-Positional sequence unless the separators are different? Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 2

do we allow indexing of non-array, non-optional
by Mike Beckerle 17 Nov '14

17 Nov '14

I don't recall whether we decided this matter or not. I would search for it myself, but I tried and failed to find anything. This is a hard topic to do searching on... no good keywords. If element 'e' has maxOccurs = 1, minOccurs = 1 (or neither are mentioned), is e[1] a valid expression? what about e[../some/pathExp/here]? If element 'e' has minOccurs= 0 maxOccurs = 1, is e[1] a valid expression, or do you access an optional element just by ../e ? Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

3 4

Re: [DFDL-WG] referencing local element declarations from path expressions between two schema documents
by Mike Beckerle 17 Nov '14

17 Nov '14

Roger, I am struggling to defend the semantics you suggest below, because I cannot find anything in the XSD spec that specifically says how path steps are interpreted. In the absence of that, other members of the DFDL workgroup argue that path steps are just references to elements, and the default xmlns definition should work exactly as it works for any QName. So if the default namespace is defined, then a name with no prefix is by default interpreted relative to that definition. Hence, you have to do this: <xs:schema xmlns="BB"... > .... <xs:key name="myKey"> <xs:selector path="a:foo/bar" xmlns=""/> ... That is to say, if you wish to refer to locally declared elements with form unqualified, you cannot have a default namespace in effect, and the above provides the way you work around this, which is you explicitly reset the default namespace to "" meaning no-namespace. So is there anything you know of in the XSD spec that clarifies this, or is this only clarified based on precedent of how it works in Xerces XSD validator, but also Schematron, XSLT, etc. I have struggled to interpret the formal semantics document about XSD. My read of it did not find clarification of this point, but I confess to not understanding it sufficiently to be sure. Thanks in advance ...mike beckerle Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> On Wed, Sep 17, 2014 at 8:30 AM, Costello, Roger L. <costello(a)mitre.org> wrote: > Hi Mike, > > > > 1. In XSD only the global stuff can be referenced. So this is not allowed: > > > > <xs:element ref="bb:b1/c1"> > > > > 2. When using xs:key the XPath expression in @path must have a namespace > prefix even if the targetNamespace is the > > default namespace. Eg. *bk:Book. *If the schema had instead set > elementFormDefault="unqualified" then the XPath expressions would not be > namespace-qualified. > > > > 3. If you embed Schematron in an XSD file, during processing the > Schematron processor will extract out the Schematron and generate an XSLT > program. At that point you are in the world of XSLT and ordinary XPath > rules apply. So if BB.xsd has declared a global element {BB}b1 which > contains {}c1 then the XPath expression to reference c1 in an instance > document is this: bb:b1/c1 (where bb has been bound to BB in the > Schematron/XSLT code). It’s irrelevant to Schematron/XSLT whether the XML > Schemas have/don’t have a default namespace declaration. > > > > /Roger > > > > *From:* Mike Beckerle [mailto:mbeckerle.dfdl@gmail.com] > *Sent:* Wednesday, September 17, 2014 4:37 AM > *To:* dfdl-wg(a)ogf.org; Costello, Roger L. > *Subject:* referencing local element declarations from path expressions > between two schema documents > > > > > > I am looking for clarification about how path steps are to be interpreted > with respect to the issue of qualified/unqualified element names, and > default or no-default namespaces. > > If there is a precedent we can follow set by other tools which embed paths > into XSD (e.g., schematron?) or even just the selector/paths things in XSD > key/uniqueness constraints then I'd like to understand it. > > These are some of the conundrums: > > > Two Schema Files: A, and B > > Schema file A has target namespace AA, and binds the AA namespace to > prefix aa, and the BB namespace to prefix bb > > Schema file B has target namespace BB, and the same prefix bindings. > > Schema B has an element named b1 which has two local element declarations > inside it named c1 and c2. > > Schema A has an element named a1 which contains an element reference to > bb:b1. > > a1 has a DFDL annotation (e.g., dfdl:assert) which has a path expression > in it. this path contains bb:b1/c1 > > When is the path correct? > > 1a) when schema B has elementFormDefault='unqualified' (that's the > default), and schema A does NOT have a default namespace. Why? the default > namespace - if it existed for schema A - would implicitly qualify the c1 > step of the path, but since c1 is a local element in schema B with element > form unqualified, it does not require (and cannot have) a namespace > specified in path steps for c1. > > 1b) when schema B has elementFormDefault='qualified', and schema A has > xmlns='BB'. Why? Because in this case the path bb:b1/c1 is equivalent to > bb:b1/bb:c1, and since c1 will have a namespace-qualified name in schema B, > that path will properly describe the path to the c1 element. > > 1c) Variation: when schema B has no target namespace (which I believe > makes elementFormDefault irrelevant), and schema A has no default > namespace. Because then c1 can't be qualfied as it lives in no namespace, > and schema A has no implied qualifiers. > > > > When is the path incorrect? > > 2a) when schema A has xmlns="BB". Why? Because the default namespace means > the bb:b1/c1 is equivalent to {BB}b1/{BB}c1, but there is no {BB}/c1 since > c1 is just a local element declaration, and B has elementFormDefault > unqualified. > > 2b) when schema B has elementFormDefault='qualified', and schema A has no > default namespace. In that case the step c1 needs a namespace qualifier > either by default, or explicitly via a prefix, but it doesn't have one. > > So I am interested in opinions on whether the above analysis is correct or > not, and especially in precedent for how this stuff works in other tools. > > I have made a big assumption above which is that if xmlns="BB", then any > path step inside something carrying that default binding is interpreted as > qualified by that namespace. I picked this because it seemed reasonable, > and it seems one can always modify one's schema to properly address > something in another schema without having to change that other schema. > I.e., it seems to have good composition properties. > > Thanks for any thoughts. > > > > > > Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | > www.tresys.com > > Please note: Contributions to the DFDL Workgroup's email discussions are > subject to the OGF Intellectual Property Policy > <http://www.ogf.org/About/abt_policies.php> > > >

1 0

OGF DFDL WG Call Agenda 2014-11-18
by Steve Hanson 17 Nov '14

17 Nov '14

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13374?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Minutes 2014-11-11
by Steve Hanson 12 Nov '14

12 Nov '14

Please find minutes from the above call at http://redmine.ogf.org/dmsf_files/13373?download= ** Updated DFDL 1.0 specification published at http://www.ogf.org/documents/GFD.207.pdf. ** Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] elementFormDefault and the form attribute on element declarations
by Steve Hanson 12 Nov '14

12 Nov '14

WG agreed that elementFormDefault and form attributes will be mentioned explicitly in section 5.1. http://redmine.ogf.org/issues/241 Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 03/11/2014 14:57 Subject: Re: [DFDL-WG] elementFormDefault and the form attribute on element declarations Section 5.1 of the spec has 'Standard XSDL namespace management' in its list of what a DFDL schema consists of. Section 5.1 does not go on explicitly to exclude 'elementFormDefault' attribute or 'form' attribute, so my reading is that both these are therefore allowed. We could expand what is meant by 'Standard XSDL namespace management'. But if we do that we should make sure that we do so in any other areas that are a bit vague. For example, we disallow complex type derivation and substitution groups so does this imply that 'final' and 'block' on complex types and elements are prohibited regardless of setting? For example, we disallow identity constraints but does that mean that 'id' can never be used on schema objects? Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 15/10/2014 23:25 Subject: [DFDL-WG] elementFormDefault and the form attribute on element declarations Sent by: dfdl-wg-bounces(a)ogf.org Neither the term "elementFormDefault" nor the term "unqualified" appear in the DFDL Spec. So we're silent about whether these are allowed or not in DFDL schemas. We should specify what of this XML Schema behavior is allowed, which I believe means add a line to each of the lists in section 5.1 of things used and not-used/reserved from XML Schema. FYI: Today the Daffodil implementation implements elementFormDefault on the xs:schema but not the form attribute on element declarations. We have tests that exercise elementFormDefault in qualified and unqualified and not specified at all (which defaults to "unqualified" behavior in XSD, so we're consistent with that.) Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: minor/typographical - fn prefix namespace not in spec
by Steve Hanson 12 Nov '14

12 Nov '14

WG agreed that the namespace URI needs adding to section 23 and linking to the fn: prefix. http://redmine.ogf.org/issues/240 Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 12/11/2014 12:04 ----- From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 28/10/2014 17:41 Subject: [DFDL-WG] minor/typographical - fn prefix namespace not in spec Sent by: dfdl-wg-bounces(a)ogf.org The URI corresponding to the xpath functions namespace http://www.w3.org/2005/xpath-functions is not in the DFDL spec. I am assuming where we are borrowing the exact XPath function behavior the namespace is the xpath functions URI, not some DFDL-specific URI. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] clarification: fn:exactly-one function
by Steve Hanson 12 Nov '14

12 Nov '14

WG agreed that fn:exactly-one() would have its description updated to match XPath 2.0 spec. Perhaps also worth updating fn:exists() and fn:empty() so their descriptions match XPath 2.0 spec. http://redmine.ogf.org/issues/239 Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Cc: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 03/11/2014 14:08 Subject: Re: [DFDL-WG] clarification: fn:exactly-one function The main purpose envisaged for fn:exactly-one() was as a guard around off-the-shelf XPath processors. In that scenario, throwing an error is probably required, as what would you do otherwise? If you wanted to return a true/false value could you just use fn:count($arg) eq 1 instead of fn:exactly-one() ? Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org> Date: 30/10/2014 10:00 Subject: [DFDL-WG] clarification: fn:exactly-one function Sent by: dfdl-wg-bounces(a)ogf.org According to the DFDL spec, fn:exactly-one returns "True if the provided sequence contains exactly one node/value," which suggests that the function would return false if the sequence contains > or < 1 node/value. The XPath description states that fn:exactly-one "returns $arg if it contains exactly one item. Otherwise, raises an error." It seems of value to have a true/false result, because something that raises an error like this can only be used as an assertion effectively. On the other hand, that's not what XPath does. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0