Mike, I have several email threads that
track the original design discussion for action 186.
You originally wrote down an algorithm
for combining statement annotations into an ordered list, but then the
next version scrapped that and proposed the current scheme. I think there
must have been some reason for this?
Are we just talking setVariable here,
or also newVariableInstance and assert as well? If so would the ordering
rules be the same within, and across, annotation points?
A couple of users have asked about short
form versions of statement annotations. Imposing an execution order is
not compatible with this. This was discussed in one of the action 186 emails.
Regards
Steve Hanson
Architect, IBM Data Format Description Language (DFDL)
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From:
Mike Beckerle <mbeckerle.dfdl@gmail.com>
To:
Steve Hanson/UK/IBM@IBMGB,
Cc:
"dfdl-wg@ogf.org"
<dfdl-wg@ogf.org>
Date:
11/09/2013 14:48
Subject:
Re: [DFDL-WG]
questions on setVariable
It is when there is more than one setVariable at one single
annotation point that I believe they should definitely execute in lexical
order. I am much less worried about across different annotation points
that get resolved together which is what the Errata was clarifying.
My preference would be this: If an element ref has 2 setVariable statements
for variables A, B, and the element declaration has setVariable statements
for variables X, Y, then ABXY, AXBY, AXYB, XYAB are all possible interleavings,
but A is always before B, and X is always before Y so execution is always
consistent with lexical order.
I want to just clarify the hole we've left by not being
tight enough about specifying the order of this evaluation of setVariables
within or across annotation points in a resolved set.
I can build a completely non-deterministic schema:
<defineVariable name="x" defaultValue="1"/>
<defineVariable name="y" defaultValue="2"/>
<defineVariable name="z" defaultValue="0"/>
<group name="coinFlip"/>
<choice>
<sequence>
<annotation><appinfo>
<dfdl:setVariable name="x" value="{ if ($y = 2) then
5/$z else 3 }"/> <!-- div by zero to create a proc error -->
<dfdl:setVariable name="y" value="1"/>
</appinfo></annotation>
<element name="nonLexicalOrder"
type="string" dfdl:inputValueCalc="{ '' }"/>
</sequence>
<element name="lexicalOrder"
type="string" dfdl:inputValueCalc="{ '' }"/>
</choice>
</group>
The above group definition illustrates how we can test
what order a particular implementation executes these setVariable statements.
You get <lexicalOrder/> or <nonLexicalOrder/>
depending on what the implementation does. It could even vary from run
to run for one implementation if the implementation is using a hash implementation
that is using different random seeds at different times. It could even
vary within a run, if this group is reused several times in the schema.
Now, why would somebody write this, I don't know, but
it does illustrate the hole in the spec. Certainly it would be better if
DFDL could not express things like this.
computing x reads the value of y. If the assignment to
y has not yet happened, then we get a processing error when we divide by
zero. This can cause backtracking, and the schema will effectively dodge
the SDE that is supposed to happen to prevent this sort of assignment-timing
stuff.
This doesn't fix the problem entirely. I think our current
'non-specification' of the precise order here even across annotation points
was a bit of a cop-out.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology
| www.tresys.com
Please note: Contributions to the DFDL Workgroup's email
discussions are subject to the OGF
Intellectual Property Policy
On Wed, Sep 11, 2013 at 6:26 AM, Steve Hanson <smh@uk.ibm.com>
wrote:
"I think the
execution order needs to be specific within the annotations at one annotation
point."
The problem is that action 186 allowed scenarios like setVariable annotations
simultaneously on an element ref, its global element and the element's
simple type (*). So there is no longer 'one annotation point', there are
multiple. That's when we formalised the concept of a 'resolved set of annotations'
for a component, and stated that for all annotations of the same kind (**)
in that set, the order of execution is not defined. We also state schema
authors can insert sequences to provide more precise control over when
annotations are executed.
Several reasons for this. Is the lexical order of elements in appinfos,
or of multiple appinfos, guaranteed in XSDL? If not then an editor could
serialize appinfos or their content in any order. We also wanted to reserve
the right to add explicit timing control in the future. And it made implementations
easier.
IBM DFDL 1.0.3 has been out in the field with this support since 1Q 2013.
Tim, can you say what the execution order is for a resolved set of setVariables
please? Or can we not tell because of the model?
(*) Must be different variables, else SDE.
(**) Asserts and discriminators with testKind 'pattern' are a different
'kind' from those with 'expression', for this purpose.
Regards
Steve Hanson
Architect, IBM Data Format Description Language (DFDL)
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From: Suman
Kalia <kalia@ca.ibm.com>
To: Mike
Beckerle <mbeckerle.dfdl@gmail.com>,
Cc: "dfdl-wg@ogf.org"
<dfdl-wg@ogf.org>,
"dfdl-wg-bounces@ogf.org"
<dfdl-wg-bounces@ogf.org>,
Steve Hanson/UK/IBM@IBMGB
Date: 10/09/2013
19:23
Subject: Re:
[DFDL-WG] questions on setVariable
Mike - In the use case hilited by Jonathan where 2 annotations are
defined on the same point. I would suggest the execution order to
be based on the lexical order i.e. the order in which annotations
appear..
Suman Kalia
IBM Canada Lab
WMB Toolkit Architect and Development Lead
Tel: 905-413-3923
T/L 313-3923
Email: kalia@ca.ibm.com
For info on Message broker
http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.html
From: Mike
Beckerle <mbeckerle.dfdl@gmail.com>
To: Steve
Hanson <smh@uk.ibm.com>,
Cc: "dfdl-wg@ogf.org"
<dfdl-wg@ogf.org>,
"dfdl-wg-bounces@ogf.org"
<dfdl-wg-bounces@ogf.org>
Date: 09/10/2013
02:14 PM
Subject: Re:
[DFDL-WG] questions on setVariable
Sent by: dfdl-wg-bounces@ogf.org
If the first setVariable executes first, that will read variable baz, which
will cause the second setVariable to SDE because you cannot set after the
variable has been read. I believe this is the right semantics for this
schema, because both these setVariable annotations are written at the same
annotation point.
If the second setVariable executes first, that will assign variable baz,
which is OK, then when the first setVariable executes, it will get the
value that was just assigned.
While the spec says you shouldn't depend on this kind of ordering stuff,
I think it is pretty unsatisfactory if the order is unspecified. Section
9.5 lays out the order of the different kinds of statement annotations
relative to each other, but doesn't say what happens between two setVariable
statements that appear at the same annotation point. I think the execution
order needs to be specific within the annotations at one annotation point.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF
Intellectual Property Policy
On Tue, Sep 10, 2013 at 1:50 PM, Steve Hanson <smh@uk.ibm.com>
wrote:
1) I'll let Tim comment more on this but a relative expression uses as
its context the current element on the stack. So using '.' on a simple
element or its type resolves to the simple element, and using '.' on a
complex element or a group child (recursively) of a complex element resolves
to the complex element. Same deal when using the '..' parent axis, so beware
- as this example shows:
<xs:element name="root">
<xs:complexType>
<xs:sequence>
<xs:element name="delim" type="xs:string"/>
<xs:element name="body">
<xs:complexType>
<xs:sequence
dfdl:separator="{../delim}">
...and not {../../delim} ! This is all implied by the way XPath works.
XPath only knows about elements.
2) I don't see why Jonathan's example could give an SDE. The defineVariables
will always be executed first. However the result is ambiguous, depending
on the order each setVariable is executed, as 'bar' could end up as 'biff'
or 'fooey'. "Schema authors can insert sequences to provide more precise
control over when variables are set" - quote from section 9.5.3 (errata
3.25).
3) To Mike's comment, the order of execution of setVariable and newVariableInstance
on the same annotation is strictly defined, see section 9.5 (errata 3.25)
Regards
Steve Hanson
Architect, IBM Data Format Description Language (DFDL)
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From: Mike
Beckerle <mbeckerle.dfdl@gmail.com>
To: "Cranford,
Jonathan W." <jcranford@mitre.org>,
Cc: "dfdl-wg@ogf.org"
<dfdl-wg@ogf.org>
Date: 10/09/2013
18:23
Subject: Re:
[DFDL-WG] questions on setVariable
Sent by: dfdl-wg-bounces@ogf.org
Those are both public-comment-class issues.
The issue of what does "." mean in various places is interesting.
It is referring to an infoset node, but how to explain exactly which one
it is....
The out-of-order issue with evaluation that you mention is definitely an
issue. The schema you gave will work or SDE depending on the order of evaluation
of the statements and the language is not precise enough to say what the
order is. I think the order of the setVariable and newVariableInstance
statements at any annotation point, must be linear first to last. The order
of setVariable and newVariableInstance combined into the resolved set of
annotations, but from multiple annotation points (e.g., an element and
a global type definition it references), those can be interleaved in any
order.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF
Intellectual Property Policy
On Tue, Sep 10, 2013 at 12:53 PM, Cranford, Jonathan W. <jcranford@mitre.org>
wrote:
These questions may go towards public comment if the answers aren't straight-forward.
Section references are against latest editor's draft.
· Section
7.9 - 5th paragraph after first example - "A dfdl:setVariable value
expression may refer to the value of this element using a relative path
value '.'. "
Which element?
If dfdl:setVariable is attached to a simpleType, then the
"element" referred to here is the element of that type, correct?
But when it's attached to group reference, sequence, or
choice, is it the parent element which contains the group reference, sequence,
or choice?
· Section
7.9 - next-to-last paragraph: "However, the order of execution
among them is not specified."
Wouldn't this be ambiguous, then?
<xs:schema>
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/">
<xs:defineVariable
name="bar" />
<xs:defineVariable
name="baz" defaultValue="biff"/>
</xs:appinfo>
</xs:annotation>
...
<xs:element name="foo">
<xs:annotation>
<xs:appinfo
source="http://www.ogf.org/dfdl/">
<xs:setVariable
ref="bar" value="{$tns:baz}"/>
<xs:setVariable
ref="baz" value="fooey"/>
</xs:appinfo>
</xs:annotation>
</xs:element>
...
</xs:schema>
If the order of execution isn't specified, isn't it ambiguous
whether tns:bar gets the value of "biff" (defaultValue of tns:baz)
or "fooey"?
Respectfully,
--
Jonathan W. Cranford
Senior Information Systems Engineer
The MITRE Corporation (http://www.mitre.org)
--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU