Mike, For setVariable statements, the IBM implementation does not do any such analysis - the order of execution is not deterministic. For defineVariable statements, the IBM implementation does act as you have described, in order to evaluate the defaultValue expressions in the 'correct' order. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Tim Kimber/UK/IBM@IBMGB, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org> Date: 11/09/2013 14:40 Subject: Fwd: [DFDL-WG] questions on setVariable Tim, When you say a "hash", is the IBM implementation implementing some sort of data-dependency analysis here? i.e., <setVariable name="x" value="{ $y }"/> <setVariable name="y" value="5"/> will execute in the opposite order because when executing the setVariable for x, it demands the value for y, and that executes the setVariable for y ?? Or is it truly just they all go into a hash table, and they come out in whatever order the hash table implementation produces them? ...mike Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Wed, Sep 11, 2013 at 7:08 AM, Tim Kimber <KIMBERT@uk.ibm.com> wrote: The model used by the IBM implementation supplies the variable-related annotations for an XSD component as a hash, so the lexical order is lost. I don't know whether it would even be possible to preserve lexical order. Suman may know the answer to that. regards, Tim Kimber, DFDL Team, Hursley, UK Internet: kimbert@uk.ibm.com Tel. 01962-816742 Internal tel. 37246742 From: Steve Hanson/UK/IBM@IBMGB To: Suman Kalia <kalia@ca.ibm.com>, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, "dfdl-wg-bounces@ogf.org" <dfdl-wg-bounces@ogf.org> Date: 11/09/2013 11:26 Subject: Re: [DFDL-WG] questions on setVariable Sent by: dfdl-wg-bounces@ogf.org "I think the execution order needs to be specific within the annotations at one annotation point." The problem is that action 186 allowed scenarios like setVariable annotations simultaneously on an element ref, its global element and the element's simple type (*). So there is no longer 'one annotation point', there are multiple. That's when we formalised the concept of a 'resolved set of annotations' for a component, and stated that for all annotations of the same kind (**) in that set, the order of execution is not defined. We also state schema authors can insert sequences to provide more precise control over when annotations are executed. Several reasons for this. Is the lexical order of elements in appinfos, or of multiple appinfos, guaranteed in XSDL? If not then an editor could serialize appinfos or their content in any order. We also wanted to reserve the right to add explicit timing control in the future. And it made implementations easier. IBM DFDL 1.0.3 has been out in the field with this support since 1Q 2013. Tim, can you say what the execution order is for a resolved set of setVariables please? Or can we not tell because of the model? (*) Must be different variables, else SDE. (**) Asserts and discriminators with testKind 'pattern' are a different 'kind' from those with 'expression', for this purpose. Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Suman Kalia <kalia@ca.ibm.com> To: Mike Beckerle <mbeckerle.dfdl@gmail.com>, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, "dfdl-wg-bounces@ogf.org" <dfdl-wg-bounces@ogf.org>, Steve Hanson/UK/IBM@IBMGB Date: 10/09/2013 19:23 Subject: Re: [DFDL-WG] questions on setVariable Mike - In the use case hilited by Jonathan where 2 annotations are defined on the same point. I would suggest the execution order to be based on the lexical order i.e. the order in which annotations appear.. Suman Kalia IBM Canada Lab WMB Toolkit Architect and Development Lead Tel: 905-413-3923 T/L 313-3923 Email: kalia@ca.ibm.com For info on Message broker http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.ht... From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson <smh@uk.ibm.com>, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, "dfdl-wg-bounces@ogf.org" <dfdl-wg-bounces@ogf.org> Date: 09/10/2013 02:14 PM Subject: Re: [DFDL-WG] questions on setVariable Sent by: dfdl-wg-bounces@ogf.org If the first setVariable executes first, that will read variable baz, which will cause the second setVariable to SDE because you cannot set after the variable has been read. I believe this is the right semantics for this schema, because both these setVariable annotations are written at the same annotation point. If the second setVariable executes first, that will assign variable baz, which is OK, then when the first setVariable executes, it will get the value that was just assigned. While the spec says you shouldn't depend on this kind of ordering stuff, I think it is pretty unsatisfactory if the order is unspecified. Section 9.5 lays out the order of the different kinds of statement annotations relative to each other, but doesn't say what happens between two setVariable statements that appear at the same annotation point. I think the execution order needs to be specific within the annotations at one annotation point. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Tue, Sep 10, 2013 at 1:50 PM, Steve Hanson <smh@uk.ibm.com> wrote: 1) I'll let Tim comment more on this but a relative expression uses as its context the current element on the stack. So using '.' on a simple element or its type resolves to the simple element, and using '.' on a complex element or a group child (recursively) of a complex element resolves to the complex element. Same deal when using the '..' parent axis, so beware - as this example shows: <xs:element name="root"> <xs:complexType> <xs:sequence> <xs:element name="delim" type="xs:string"/> <xs:element name="body"> <xs:complexType> <xs:sequence dfdl:separator="{../delim}"> ...and not {../../delim} ! This is all implied by the way XPath works. XPath only knows about elements. 2) I don't see why Jonathan's example could give an SDE. The defineVariables will always be executed first. However the result is ambiguous, depending on the order each setVariable is executed, as 'bar' could end up as 'biff' or 'fooey'. "Schema authors can insert sequences to provide more precise control over when variables are set" - quote from section 9.5.3 (errata 3.25). 3) To Mike's comment, the order of execution of setVariable and newVariableInstance on the same annotation is strictly defined, see section 9.5 (errata 3.25) Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: "Cranford, Jonathan W." <jcranford@mitre.org>, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org> Date: 10/09/2013 18:23 Subject: Re: [DFDL-WG] questions on setVariable Sent by: dfdl-wg-bounces@ogf.org Those are both public-comment-class issues. The issue of what does "." mean in various places is interesting. It is referring to an infoset node, but how to explain exactly which one it is.... The out-of-order issue with evaluation that you mention is definitely an issue. The schema you gave will work or SDE depending on the order of evaluation of the statements and the language is not precise enough to say what the order is. I think the order of the setVariable and newVariableInstance statements at any annotation point, must be linear first to last. The order of setVariable and newVariableInstance combined into the resolved set of annotations, but from multiple annotation points (e.g., an element and a global type definition it references), those can be interleaved in any order. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy On Tue, Sep 10, 2013 at 12:53 PM, Cranford, Jonathan W. < jcranford@mitre.org> wrote: These questions may go towards public comment if the answers aren't straight-forward. Section references are against latest editor's draft. · Section 7.9 - 5th paragraph after first example - "A dfdl:setVariable value expression may refer to the value of this element using a relative path value '.'. " Which element? If dfdl:setVariable is attached to a simpleType, then the "element" referred to here is the element of that type, correct? But when it's attached to group reference, sequence, or choice, is it the parent element which contains the group reference, sequence, or choice? · Section 7.9 - next-to-last paragraph: "However, the order of execution among them is not specified." Wouldn't this be ambiguous, then? <xs:schema> <xs:annotation> <xs:appinfo source="http://www.ogf.org/dfdl/"> <xs:defineVariable name="bar" /> <xs:defineVariable name="baz" defaultValue="biff"/> </xs:appinfo> </xs:annotation> ... <xs:element name="foo"> <xs:annotation> <xs:appinfo source="http://www.ogf.org/dfdl/"> <xs:setVariable ref="bar" value="{$tns:baz}"/> <xs:setVariable ref="baz" value="fooey"/> </xs:appinfo> </xs:annotation> </xs:element> ... </xs:schema> If the order of execution isn't specified, isn't it ambiguous whether tns:bar gets the value of "biff" (defaultValue of tns:baz) or "fooey"? Respectfully, -- Jonathan W. Cranford Senior Information Systems Engineer The MITRE Corporation (http://www.mitre.org) -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU