- dfdl-wg - lists.ogf.org

Re: [DFDL-WG] Fw: setVariable with no value set - supported?
by Steve Hanson 26 Nov '12

26 Nov '12

Updated below to clarify defineVariable and newVariableInstance. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org, Cc: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Mark Frost/UK/IBM, Tim Kimber/UK/IBM@IBMGB, Shane Cartledge/Toronto/IBM@IBMCA, Anita Leung/Toronto/IBM@IBMCA Date: 22/11/2012 12:20 Subject: Re: [DFDL-WG] Fw: setVariable with no value set - supported? Proposal to allow empty string to be used as a value for DFDL variables of type xs:string and xs:hexBinary (or derived from them) without resorting to using an expression. To set a default value or value to empty string use the attribute syntax, eg: <dfdl:defineVariable name="myVar" type="xs:string" defaultValue=""/> <dfdl:newVariableInstance ref="myVar" defaultValue=""/> <dfdl:setVariable ref="myVar" value=""/> The following are legal and leave the variable without a default value. <dfdl:defineVariable name="myVar" type="xs:string" /> <dfdl:defineVariable name="myVar" type="xs:string"></ dfdl:defineVariable> The following are legal and the default value is obtained from the defineVariable. <dfdl:newVariableInstance ref="myVar" /> <dfdl:newVariableInstance ref="myVar"></ dfdl:newVariableInstance> The following are schema definition errors - they do not set the value to anything, which is the purpose of setVariable. This allows us to apply a semantic to these syntaxes in a future DFDL release if we need to (eg, unsets the value). <dfdl:setVariable ref="myVar"></dfdl:setVariable> <dfdl:setVariable ref="myVar"/> Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Cc: Mark Frost/UK/IBM, Tim Kimber/UK/IBM@IBMGB Date: 21/11/2012 09:48 Subject: Re: [DFDL-WG] Fw: setVariable with no value set - supported? Here's what Sandy had to say on this. He raised a good point about the future possibility of unsetting a variable using the empty syntax. Steve Hanson Hi Sandy - please can you ping back when free ? Ta! Sandy (S.) Gao hi steve Steve Hanson Hi ◄ From DFDL WG call today... ◄ 2. What is correct syntax for setting the value of a variable to empty string? DFDL variables allow both XSDL lexical form and DFDL expressions as values, so it seems plausible that <dfdl:setVariable ref="x" value=""/> or even <dfdl:setVariable ref="x"/> is an allowed syntax for setting the value of a string or hexBinary variable to empty string ◄ What's your opinion here? ◄ Syntax for setting a DFDL variable to empty string. Sandy (S.) Gao the first looks reasonable. if you want to distinguish between an unset value / null value vs. empty value, then you may not want to use the second. in xml, there's a difference between an attribute that's not present and an attribute with empty value. ◄ unless you are using element content as the value for the second case. Steve Hanson That's the issue - DFDL treats the attr form and element form as equivalent Sandy (S.) Gao so for non-empty value, you could <setV ... value="abc"/> or <setV ...>abc</setV>? Steve Hanson correct Sandy (S.) Gao ic... then i suppose you need to be consistent and allow both of those forms. ◄ but can someone specify a null value? or unset a variable? Steve Hanson null not allowed ◄ I don't think we allow unsetting either ◄ Good points though Sandy (S.) Gao if a value is required, then i don't see any issue in this approach. Steve Hanson ok - I will raise the unset issue, as we don't want to box ourselves in for future ◄ thanks Sandy Sandy (S.) Gao np... i guess you'll need to add a new element e.g. <unsetValue> or a special value <setV...>##unset</setV> to support it. Steve Hanson yes - if we don't allow the empty element + missing attr to mean that Sandy (S.) Gao exactly Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Date: 19/11/2012 15:11 Subject: Re: [DFDL-WG] Fw: setVariable with no value set - supported? Here is the way I would set a variable to the empty string. <dfdl:setVariable ref="someQName">{ "" }</dfdl:setVariable> Note that the value is an expression and so must have the {...} surround, the expression returns an empty string as its value. If you don't mind quoting hell, then <dfdl:setVariable ref="someQName" value='{ "" }'/> would also work. The DFDL validator doesn't catch the error about the value attribute being missing because both the above forms are accepted and XML Schema's validation can't enforce an OR of "you have to have an element child value OR the value attribute". This check has to be done by the DFDL implementation itself, not just XSD validation of the DFDL annotations. On Mon, Nov 19, 2012 at 9:34 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: See below... Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 19/11/2012 14:23 ----- From: Steve Hanson/UK/IBM To: Mark Frost/UK/IBM, Cc: Shane Cartledge/Toronto/IBM@IBMCA, Anita Leung/Toronto/IBM@IBMCA Date: 16/11/2012 11:46 Subject: Re: setVariable with no value set - supported? The specific example that Shane gives is invalid anyway as you can't set the value of one of the pre-defined DFDL variables. Assuming a realistic example like: <dfdl:setVariable ref="myVar"/> I'm wondering how you would set empty string for an element of type xs:string or xs:hexBinary. <dfdl:setVariable ref="myVar" value=""/> But why isn't that the same as <dfdl:setVariable ref="myVar"></dfdl:setVariable> which in XML rules is equivalent to: <dfdl:setVariable ref="myVar"/> I will raise with the DFDL WG. Thanks Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mark Frost/UK/IBM To: Shane Cartledge/Toronto/IBM@IBMCA, Steve Hanson/UK/IBM@IBMGB, Cc: Anita Leung/Toronto/IBM@IBMCA Date: 16/11/2012 11:19 Subject: Re: setVariable with no value set - supported? Hi Shane, While the spec isn't super-specific, I'd interpret it to mean "you must give exactly one of the value attribute or element value" Spec Section 7.9 : "The syntax supports both a dfdl:value attribute and the 'value' being specified by the element value. Only one or the other may be present. (Schema definition error otherwise.)" Steve: do you think altering the spec wording above to something like " Exactly one of the dfdl:value or the element value must be present. (Schema definition error otherwise.)" Regards, Mark Frost _____________________________________ MP 211, IBM Hursley, Winchester, SO21 2JN Phone: (01962) 817009 or x247009 Email: frostmar(a)uk.ibm.com From: Shane Cartledge/Toronto/IBM@IBMCA To: Mark Frost/UK/IBM@IBMGB, Cc: Anita Leung/Toronto/IBM@IBMCA Date: 15/11/2012 22:56 Subject: setVariable with no value set - supported? Hi Mark, Do you know if DFDL should supports a setVariable with no value set? Thought this may have already come up with your runtime testing. For example: <dfdl:setVariable ref="dfdl:binaryFloatRep"/> The DFDL specification is not clear on this point, and the DFDL validator does not flag it as an error. Thanks. Shane Cartledge Business Integration Tooling Phone: 1-905-413-5171, Tie Line: 313-5171 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Broadcast: OGF DFDL WG Call (8 Jan 15:00 GMT in DE3S09)
by Steve Hanson 26 Nov '12

26 Nov '12

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg

1 0

AUTO: Sunil Dandamudi is out of the office (returning 11/27/2012)
by Sunil Dandamudi 23 Nov '12

23 Nov '12

I am out of the office until 11/27/2012. Note: This is an automated response to your message "dfdl-wg Digest, Vol 77, Issue 34" sent on 11/23/2012 10:42:15. This is the only notification you will receive while this person is away.

1 0

OGF DFDL WG Call Minutes 2012-11-20
by Steve Hanson 23 Nov '12

23 Nov '12

Please find minutes from the above call on Redmine at http://redmine.ogf.org/dmsf_files/12967?download= Errata document is now at v011 (draft) here: http://redmine.ogf.org/dmsf_files/8104?download= ** No new documents will be added to GridForge ** Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] Fw: setVariable with no value set - supported?
by Steve Hanson 22 Nov '12

22 Nov '12

Proposal to allow empty string to be used as a value for DFDL variables of type xs:string and xs:hexBinary (or derived from them) without resorting to using an expression. To set empty string use the attribute syntax, eg: <dfdl:defineVariable name="myVar" type="xs:string" defaultValue=""/> <dfdl:newVariableInstance ref="myVar" defaultValue=""/> <dfdl:setVariable ref="myVar" value=""/> The following are schema definition errors as they do not set the value to anything. <dfdl:setVariable ref="myVar"></dfdl:setVariable> <dfdl:setVariable ref="myVar"/> That allows us to apply a semantic to these syntaxes in a future DFDL release if we need to (eg, unsets the value). Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Cc: Mark Frost/UK/IBM, Tim Kimber/UK/IBM@IBMGB Date: 21/11/2012 09:48 Subject: Re: [DFDL-WG] Fw: setVariable with no value set - supported? Here's what Sandy had to say on this. He raised a good point about the future possibility of unsetting a variable using the empty syntax. Steve Hanson Hi Sandy - please can you ping back when free ? Ta! Sandy (S.) Gao hi steve Steve Hanson Hi ◄ From DFDL WG call today... ◄ 2. What is correct syntax for setting the value of a variable to empty string? DFDL variables allow both XSDL lexical form and DFDL expressions as values, so it seems plausible that <dfdl:setVariable ref="x" value=""/> or even <dfdl:setVariable ref="x"/> is an allowed syntax for setting the value of a string or hexBinary variable to empty string ◄ What's your opinion here? ◄ Syntax for setting a DFDL variable to empty string. Sandy (S.) Gao the first looks reasonable. if you want to distinguish between an unset value / null value vs. empty value, then you may not want to use the second. in xml, there's a difference between an attribute that's not present and an attribute with empty value. ◄ unless you are using element content as the value for the second case. Steve Hanson That's the issue - DFDL treats the attr form and element form as equivalent Sandy (S.) Gao so for non-empty value, you could <setV ... value="abc"/> or <setV ...>abc</setV>? Steve Hanson correct Sandy (S.) Gao ic... then i suppose you need to be consistent and allow both of those forms. ◄ but can someone specify a null value? or unset a variable? Steve Hanson null not allowed ◄ I don't think we allow unsetting either ◄ Good points though Sandy (S.) Gao if a value is required, then i don't see any issue in this approach. Steve Hanson ok - I will raise the unset issue, as we don't want to box ourselves in for future ◄ thanks Sandy Sandy (S.) Gao np... i guess you'll need to add a new element e.g. <unsetValue> or a special value <setV...>##unset</setV> to support it. Steve Hanson yes - if we don't allow the empty element + missing attr to mean that Sandy (S.) Gao exactly Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Date: 19/11/2012 15:11 Subject: Re: [DFDL-WG] Fw: setVariable with no value set - supported? Here is the way I would set a variable to the empty string. <dfdl:setVariable ref="someQName">{ "" }</dfdl:setVariable> Note that the value is an expression and so must have the {...} surround, the expression returns an empty string as its value. If you don't mind quoting hell, then <dfdl:setVariable ref="someQName" value='{ "" }'/> would also work. The DFDL validator doesn't catch the error about the value attribute being missing because both the above forms are accepted and XML Schema's validation can't enforce an OR of "you have to have an element child value OR the value attribute". This check has to be done by the DFDL implementation itself, not just XSD validation of the DFDL annotations. On Mon, Nov 19, 2012 at 9:34 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: See below... Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 19/11/2012 14:23 ----- From: Steve Hanson/UK/IBM To: Mark Frost/UK/IBM, Cc: Shane Cartledge/Toronto/IBM@IBMCA, Anita Leung/Toronto/IBM@IBMCA Date: 16/11/2012 11:46 Subject: Re: setVariable with no value set - supported? The specific example that Shane gives is invalid anyway as you can't set the value of one of the pre-defined DFDL variables. Assuming a realistic example like: <dfdl:setVariable ref="myVar"/> I'm wondering how you would set empty string for an element of type xs:string or xs:hexBinary. <dfdl:setVariable ref="myVar" value=""/> But why isn't that the same as <dfdl:setVariable ref="myVar"></dfdl:setVariable> which in XML rules is equivalent to: <dfdl:setVariable ref="myVar"/> I will raise with the DFDL WG. Thanks Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mark Frost/UK/IBM To: Shane Cartledge/Toronto/IBM@IBMCA, Steve Hanson/UK/IBM@IBMGB, Cc: Anita Leung/Toronto/IBM@IBMCA Date: 16/11/2012 11:19 Subject: Re: setVariable with no value set - supported? Hi Shane, While the spec isn't super-specific, I'd interpret it to mean "you must give exactly one of the value attribute or element value" Spec Section 7.9 : "The syntax supports both a dfdl:value attribute and the 'value' being specified by the element value. Only one or the other may be present. (Schema definition error otherwise.)" Steve: do you think altering the spec wording above to something like " Exactly one of the dfdl:value or the element value must be present. (Schema definition error otherwise.)" Regards, Mark Frost _____________________________________ MP 211, IBM Hursley, Winchester, SO21 2JN Phone: (01962) 817009 or x247009 Email: frostmar(a)uk.ibm.com From: Shane Cartledge/Toronto/IBM@IBMCA To: Mark Frost/UK/IBM@IBMGB, Cc: Anita Leung/Toronto/IBM@IBMCA Date: 15/11/2012 22:56 Subject: setVariable with no value set - supported? Hi Mark, Do you know if DFDL should supports a setVariable with no value set? Thought this may have already come up with your runtime testing. For example: <dfdl:setVariable ref="dfdl:binaryFloatRep"/> The DFDL specification is not clear on this point, and the DFDL validator does not flag it as an error. Thanks. Shane Cartledge Business Integration Tooling Phone: 1-905-413-5171, Tie Line: 313-5171 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Action 192: URN scheme - was: Re: new action item needed: DFDL URN specification
by Mike Beckerle 20 Nov '12

20 Nov '12

I'd like to add a sub-space for implementations to use for their implementation-specific things. Nice thing about a URN is that it can specify a structured scheme for this: I suggest "imp" as a sub-specifier for implementation-specific URNs. E.g., urn:ogf:dfdl:2011:imp:... What goes after imp is an identifier of an implementation. E.g., - urn:ogf:dfdl:2011:imp:opensource.ncsa.illinois.edu:... - urn:ogf:dfdl:2011:imp:ibm.com:... and beneath that is anything that implementation needs/wants. E.g., a namespace for extension properties that are non-standard, a version ID for sub-versions of that implementation, etc. etc. This allows an implementation to define extension properties, and have other implementations give warnings that those are detected, but not understood. ...mikeb On Mon, Oct 29, 2012 at 11:37 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: > Using the OGF's URN registration document [GFD.191] as a reference, I > suggest: > *Type* *URL* *Proposed URN* common root urn:ogf:dfdl xmlns attribute > http://www.ogf.org/dfdl/dfdl-1.0/ urn:ogf:dfdl:2011 source attribute > http://www.ogf.org/dfdl/ urn:ogf:dfdl > > [GFD.191] http://www.ogf.org/documents/GFD.191.pdf > > Regards > > Steve Hanson > Architect, Data Format Description Language (DFDL) > Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> > IBM SWG, Hursley, UK* > **smh(a)uk.ibm.com* <smh(a)uk.ibm.com> > tel:+44-1962-815848 > > > > From: Suman Kalia <kalia(a)ca.ibm.com> > To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, > Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org > Date: 25/10/2012 17:15 > Subject: Re: [DFDL-WG] new action item needed: DFDL URN > specification > Sent by: dfdl-wg-bounces(a)ogf.org > ------------------------------ > > > > I tend to agree that specifying urn for namespaces is better choice and > URL should be used for schema locations; however convention of specifying > URL for namespaces is long established some implementations use the > namespace URL to return the actual schema. When I try to access the > namespace URL for XML schema , it gives me reference to the document but > not schema.. > > > > Suman Kalia > IBM Canada Lab > WMB Toolkit Architect and Development Lead > Tel: 905-413-3923 T/L 313-3923 > Email: kalia(a)ca.ibm.com > > For info on Message broker * > ** > http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h… > *<http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h…> > > > > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> > To: dfdl-wg(a)ogf.org, > Date: 10/25/2012 08:18 AM > Subject: Re: [DFDL-WG] new action item needed: DFDL URN > specification > Sent by: dfdl-wg-bounces(a)ogf.org > ------------------------------ > > > > I wanted to add to this discussion the w3c blog page that highlights the > problem of the http-based naming scheme. There are many articles about > this, this is just one of them.* > ** > **http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_dtd_traffic/*<http://www.w3.org/blog/systeam/2008/02/08/w3c_s_excessive_dtd_traffic/> > > On Wed, Oct 24, 2012 at 10:35 AM, Mike Beckerle <*mbeckerle.dfdl(a)gmail.com > * <mbeckerle.dfdl(a)gmail.com>> wrote: > > Introduction: > > Using URLs as identifiers has caused no end of problems. E.g., in DFDL we > have *http://www.ogf.org/dfdl/dfdl-1.0/*<http://www.ogf.org/dfdl/dfdl-1.0/>as an identifier. W3C has > *http://www.w3.org/2001/XMLSchema* <http://www.w3.org/2001/XMLSchema> and > others. > > W3C has badly regretted establishing this convention, as they have farms > of servers that do nothing but quickly return 404 errors to save > network-aware applications the network-timeout delay that would otherwise > occur. > > There are parties interested in exploiting DFDL who want DFDL schemas to > NOT contain network URLs because it simply creates a concern about network > access whenever a DFDL schema is inspected/used. > > Pre-Proposal: > > The new way to do this is with URNs which would look roughly like this: > urn:ogf:dfdl:standard:dfdl-1.0. The whole point is that some other > mechanism is used to establish correspondences between these and any > resources in file systems, networks, or built-in to implementations. One > such mechanism is called XML Catalog. > > The point is that it is a name in a managed namespace which cannot be > confused with a network protocol URL. > > OGF is already establishing urn:ogf, and an ogf subgroup has already > proposed urn:ogf:network for network resources. DFDL schemas aren't network > resources so we don't want to be a substructure underneath network. > > Some other mechanism is used to establish correspondences between these > and any resources in file systems, networks, or built-in to > implementations. One such mechanism is called XML Catalog. > > Summary: > > An action item should be to specify DFDL urn, submit to OGF as a proposed > namespace, and then produce errata/spec changes to specify its use. > > This requires a small design activity to specify a scheme for the > sub-structure of the DFDL URNs (i.e., scheme for the stuff after > urn:ogf:dfdl:...) where we want standard identifiers for versions of the > standard, but we probably also want a few other things (e.g., I would like > a space for implementations to identify themselves, i.e., an > implementation-specific sub-area within our URNs.) > > Our existing URLs can be compatible (deprecated) practice vs the preferred > URNs. > > > > > > > -- > Mike Beckerle | OGF DFDL WG Co-Chair > Tel: *781-330-0412* <781-330-0412> > > > > > -- > Mike Beckerle | OGF DFDL WG Co-Chair > Tel: 781-330-0412 > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > *https://www.ogf.org/mailman/listinfo/dfdl-wg*<https://www.ogf.org/mailman/listinfo/dfdl-wg> > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > https://www.ogf.org/mailman/listinfo/dfdl-wg > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

2 1

Action 186: Re: DFDL Statement Evaluation Timing (Assert, Discriminator, SetVariable, NewVariableInstance)
by Mike Beckerle 20 Nov '12

20 Nov '12

(Missed the CC of the whole WG. Here it is.) On Tue, Nov 20, 2012 at 6:49 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: > Hi Mike > > I have incorporated all this into Errata v011. A couple of comments: > > - You use 'parse error' in a few places where correct term is 'processing > error' > I try to say parse error in a parsing specific context, and processing error when something applies both parsing/unparsing, but obviously I blow it sometimes. Doesn't the spec say a parse error is a parse-specific processing error? > - I think your new section 7.10 should be in section 9, as it is talking > about order of execution and consequences of failure which are introduced > in section 9. > Sounds fine to me. I just picked one possible logical spot. > > - Regarding "*It is also a processing error to read any variables defined > by any dfdl:newVariableInstance statement evaluations that failed, or to > read any variables set by any dfdl:setVariable statements that failed.* ", > can you give a scenario where the processor can get into this situation? > > Sure. I was just having to implement this the other day, which is where this notion came from. It certainly requires scrutiny. Imagine a big complex sequence type. There's a discriminator, but it involves a complex expression, perhaps more than one field is involved in the calculation (e.g., if either this or that, then we're on a B-type record) however, the structure has substantial parse complexity beyond just identifying the two fields. Real work to do to parse it. The discriminator expression could be expressed in terms of variables just because it is a complex calculation. I think this will be common in XPath since it has no native local variables in the expression language people will use DFDL variables just to avoid having to repeat large complex XPath sub-expressions. That's ok, because if the discriminator calculation is to succeed, then those setVariable expressions would also have to succeed just as if the discriminator were expressed as a bigger more complex nested expression without variables. But, there could also be OTHER setVariable statements in the mix, unrelated to evaluating the discriminator. Those might fail if the whole parse failed. The parts of the structure they depend on might not have been successfully parsed at all. We don't want to require implementations to know which setVariable statements are needed for the discriminator and which are not, so we have this simpler strategy where we can attempt them all. If one fails, we save the error. Then when we get to evaluating the discriminator, it can still succeed, i.e., evaluate to true and resolve the PoU, so long as it doesn't need any of the failed setVariables. Then we fail with the saved error from whichever setVariable statement failed. ...mikeb

1 0

First draft of DFDL Errata v011
by Steve Hanson 20 Nov '12

20 Nov '12

Please find attached the first draft of Errata v011. This incorporates all DFDL spec errata raised since v010 (June 22 2012) until the last DFDL WG call (November 13 2012). Change tracking: I accepted all changes from v010 so the changes you see are all due to v011. I would appreciate a review for correctness and omissions. Please note that some of the new errata have been best handled by updating existing related errata, so please read from start. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Action 186: Re: DFDL Statement Evaluation Timing (Assert, Discriminator, SetVariable, NewVariableInstance)
by Mike Beckerle 20 Nov '12

20 Nov '12

I had the action to create the final errata wording for this item (Action 186). Last action issue was to change wording to insure it was all dealing with combined annotations uniformly, and change "annotated construct" to "schema component". I've done that, but also reorganized this as errata on specific sections of the spec consistent with the rest of our errata, and turned the 'ordering' section into numbered lists for clarity. ------------------------------------------------------- * Errata section 6.2. Clarification:* At any single annotation point of the schema, there can be only one format annotation (dfdl:format, dfdl:element, dfdl:sequence, dfdl:choice, dfdl:group, dfdl:simpleType). *Errata section 3. Glossary entries* *Glossary*: Annotation point: A location within a DFDL schema where DFDL annotation elements are allowed to appear. Note that annotation point is defined in section 6.2, but it belongs also in the glossary. * Glossary*: DFDL Statement annotations, or just DFDL Statements, are the annotation elements dfdl:assert, dfdl:discriminator, dfdl:setVariable, and dfdl:newVariableInstance. *Glossary*: DFDL Defining Annotations are the annotation elements dfdl:defineFormat, dfdl:defineVariable, and dfdl:defineEscapeScheme *Glossary*: DFDL Format Annotations are the annotation elements dfdl:format, dfdl:element, dfdl:simpleType, dfdl:group, dfdl:sequence, and dfdl:choice. *Glossary*: Remove existing definition for "Format Annotations" *Glossary*: Physical layer - revised to: "Physical Layer - A DFDL Schema adds DFDL annotations onto an XSDL language schema. The annotations describe the physical representation or physical layer of the data." * Glossary*: *Combined annotations*: When DFDL annotations appear on a group reference and the sequence or choice of the referenced global group, or appear among an element reference, an element declaration, and its type definition, then they are combined together and the resulting set of annotations is referred to as the *combined annotations* for the schema component.* Errata Section 7.3.1: *Remove "DFDL asserts can be placed on components within a DFDL model." as this is made more specifically subsequently. Replace "More than one dfdl:assert may be used at an annotation point. The dfdl:asserts will be evaluated in the order defined in the schema." with "If the combined annotations for a schema component contain multiple dfdl:assert statements, then those with testKind='pattern' are executed before those with testKind='expression' (the default). However, within each group the order of execution among them is not specified. Schema authors can insert sequences to control the timing of evaluation of statements more precisely. See Section REF: *Evaluation Order for Statement Annotations.*" Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. * ** Errata Section 7.3.1, and 7.4.1: Assert and Discriminator Placement* Correct description to be consistent with: dfdl:assert and dfdl:discriminator can be placed as annotations on sequence, choice, group references, local and global element declarations, element references, and simple type definitions. *Errata Section 7.4.1:* Replace: "Any one annotation point can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." with: "The combined annotations for a schema component can contain only a single dfd:discriminator or one or more dfdl:asserts, but not both. It is a schema definition error otherwise." Property testPattern: Add this to the description: Note that the pattern is used to match against the entire representation of the component; hence, the framing (including initiators) is visible to the pattern. It is a schema definition error if dfdl:alignment is not 1 . It is a schema definition error if dfdl:leadingSkip is not 0. It is a schema definition error if dfdl:encoding is not defined. *Errata Section 7.8 newVariableInstance* dfdl:newVariableInstance can be placed as an annotation on sequence, choice, and group references. Replace: "It is a schema definition error to have more than one newVariableInstances for the same variable at any given point in the document." with "The combined annotations for any schema component can contain multiple dfdl:newVariableInstance annotations, but they must each refer to a different variable. It is a schema definition error otherwise." *Errata Section 7.9: setVariable * Correct language to be consistent with: "dfdl:setVariable may be placed as an annotation on sequence, choice, group references, local and global element declarations for elements of simple type, element references to elements of simple type, and simple type definitions. " Replace "It is a schema definition error to have more than one dfdl:setVariable for the same variable at any given point in the document." with "The combined annotations for any schema component can contain multiple dfdl:setVariable annotations, but they must each refer to a different variable. It is a schema definition error otherwise." * Errata: New Section 7.10: Evaluation Order for Statement Annotations *Of the combined annotations for a schema component, some are statement annotations and the order of their evaluation relative to the actual processing of the schema component itself (parsing or unparsing per its format annotations) is as given in the ordered lists below. Implementations are free to optimize by recognizing and executing discriminators or assertions with testKind='expression' earlier so long as the resulting behavior is consistent with what results from the this description: For elements: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. Processing of the element schema component itself (as per format annotations) 4. dfdl:setVariable(s) - See note below. 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3, or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) For sequences and choices: 1. dfdl:discriminator with testKind='pattern' (parsing only) 2. dfdl:assert(s) with testKind='pattern' (parsing only) 3. dfdl:newVariableInstance(s) 4. dfdl:setVariable(s) 5. dfdl:discriminator with testKind='expression' (parsing only) - Evaluated even on failure of steps 3 or 4. See note below. 6. dfdl:assert(s) with testKind='expression' (parsing only) 7. Processing of the sequence/choice component itself (as per format annotations) * Note on Discriminators with testKind='expression' *When parsing, an attempt to evaluate a discriminator must be made even if preceding statements or the parse of the schema component ended in a parse error. This is because a discriminator's expression could evaluate to true thereby resolving a point of uncertainty even if the complete parsing of the construct ultimately caused a parse error. Such discriminator evaluation has access to the DFDL Infoset of the attempted parse as it existed immediately before detecting the parse failure. Attempts to reference parts of the DFDL Infoset that do not exist are parse errors. It is also a parse error to read any variables defined by any dfdl:newVariableInstance statement evaluations that failed, or to read any variables set by any dfdl:setVariable statements that failed. * Note on the dfdl:setVariable Statement for Elements* The combined dfdl:setVariable statements for an element schema com ponent (combined from an element reference, element declaration, and base and derived simpleType definitions) are executed after the parsing of the element. This implies that these variables are set after the evaluation of expressions corresponding to any computed DFDL format properties for the same schema component, and so the variables may not be referenced from expressions that compute DFDL properties. That is, if an expression is used to provide the value of a format property (such as dfdl:terminator, or dfdl:byteOrder), the evaluation of that format property expression occurs before any dfdl:setVariable annotation from the combined annotations for that schema component are executed; hence, the expression providing the value of the format property may not reference the variable. Schema authors can insert sequences to provide more precise control over when variables are set.

2 5

Action 189 - revise precedence order section
by Mike Beckerle 20 Nov '12

20 Nov '12

I suggested revising the precedence order section. So, I spent a few hours trying to reinvent the precedence order section along the lines of the grammar version 007. I failed. I kept ending up with a structure not too different from what exists already in the spec. The thought process goes something like this: - Create a tree corresponding to the grammar - Annotate the leaves with every relevant property. - Annotate the interior nodes of the tree with the properties needed just to get to that node of the tree, or that appear beneath that node down multiple paths. - And finally, don't repeat things. The problem: the grammar productions and terminals don't provide enough ways to share to avoid lots of duplication. The grammar leaves out exactly the details needed - representation (the grammar regions often have both a textual and a binary personality that are quite different), and length (the grammar regions are mostly silent about lengthKind). You end up wanting to create a big class diagram and have each node of the tree associated with multiple classes. It's a mixin-style multiple-inheritance scheme. It's not very interesting because so many things inherit so many properties (because they have both textual and binary forms, and work with different length kinds.) If you approximate this by what you can create in a nested hierarchy, you have to repeat some properties, but you end up being not very different from the breakdown of what is called out in the precedence section of the document as written currently..... voila... full circle. I went round and round this enough times to convince myself that whatever I was producing wasn't adding value over what is already there in the precedence section as written today. I also convinced myself that the precedence section as written, does add substantial value exactly because it is NOT organized along the lines of the grammar, but orthogonally to it. I suggest we close out Action 189. I think it will not pan out. There may be other adjustments to be made to the precedence section, but the rewrite along the lines of the grammar is not an idea that will pan out. ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

1 0