Re: [DFDL-WG] Unparsing and choices - (related - Re: Fw: Action 233 (deferred) - "byte order not sufficient..." - draft document on experience with binary format MIL-STD-2045
We'll I am a bit concerned if we cannot generally implement an asymetric parser and unparser inside the same schema. That is, if you have calculated element named C, then I think one should be able to have hidden element A which is used only when parsing (C computed from A), and hidden element B which is used only when unparsing, (B computed from C), where B is non-existing on parse, and A is non-existing on unparse. I think this trick may work: <!-- hidden. Found when parsing, not in infoset when unparsing --> <element name="A" type="xs:string" minOccurs="0" dfdl:occursCountKind=...whatever.. /> <!-- boolean. Tells whether we're parsing or unparsing based on whether A is in infoset --> <dfdl:newVariableInstance ref="areParsing" value ="{ fn:exists(../A) }"/> <!-- hidden. Length 0 when parsing. Unparsing gets length and value from C --> <element name="B" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="{ if ($areParsing) then 0 else fn:string-length(../C) }" dfdl:outputValueCalc="{ ../C }" /> <!-- not hidden --> <element name="C" type="xs:string" dfdl:inputValueCalc="{ ../A }" /> On Mon, Jul 14, 2014 at 3:23 PM, Steve Hanson <smh@uk.ibm.com> wrote:
The proposal for action 261 is that the dfdl:length expression is evaluated, so that will stop what you propose from working.
For the MIL-STD-2045 'fixedOrTerminated' case then the sequence suggestion in the other email is much simpler.
Regards
Steve Hanson Architect, *IBM DFDL* <http://www.ibm.com/developerworks/library/se-dfdl/index.html> Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> IBM SWG, Hursley, UK *smh@uk.ibm.com* <smh@uk.ibm.com> tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org> Date: 14/07/2014 19:51 Subject: Re: Unparsing and choices - (related - Re: [DFDL-WG] Fw: Action 233 (deferred) - "byte order not sufficient..." - draft document on experience with binary format MIL-STD-2045 ------------------------------
There has to be a way to get zero-length on parsing, but an outputValueCalc string on unparsing.
<defineVariable name="dummyLength" defaultValue="0" external='true'/>
<element name="outputRaw" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="{ $tns:dummyLength }" dfdl:outputValueCalc="{.....}" />
Would this work? The length expression isn't used when unparsing I believe in that some other elements are supposed to be storing the length information. That's the case here though the length value information isn't being stored, what is being stored is a way to determine the length.
Maybe there's something simpler?
On Mon, Jul 14, 2014 at 1:42 PM, Steve Hanson <*smh@uk.ibm.com* <smh@uk.ibm.com>> wrote: Mike
Section 17 of the spec says "It is a schema definition error if dfdl:inputValueCalc and dfdl:outputValueCalc are specified on the same element. " So outputRaw will give an SDE.
Regards
Steve Hanson Architect, *IBM DFDL* <http://www.ibm.com/developerworks/library/se-dfdl/index.html> Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> IBM SWG, Hursley, UK *smh@uk.ibm.com* <smh@uk.ibm.com> tel:*+44-1962-815848* <%2B44-1962-815848>
From: Mike Beckerle <*mbeckerle.dfdl@gmail.com* <mbeckerle.dfdl@gmail.com>> To: Steve Hanson/UK/IBM@IBMGB, Cc: "*dfdl-wg@ogf.org* <dfdl-wg@ogf.org>" <*dfdl-wg@ogf.org* <dfdl-wg@ogf.org>> Date: 14/07/2014 16:43 Subject: Unparsing and choices - (related - Re: [DFDL-WG] Fw: Action 233 (deferred) - "byte order not sufficient..." - draft document on experience with binary format MIL-STD-2045 ------------------------------
Breaking out this one issue for a separate discussion thread.
Consider the choice below. There is a choice of two different elements named raw1 and raw2, then a value element which is computed based on one or the other of the alternatives.
<choice> <!-- length kind pattern is needed to bound length to max of 99 --> <element name="raw1" type="xs:string" dfdl:lengthKind='pattern' dfdl:lengthPattern="[^\x7F]{0,99}" dfdl:terminator="%DEL;"/> <element name="raw2" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="100"/> </choice> <element name='value' type='xs:string' dfdl:inputValueCalc='{ if (fn:exists( ../raw1 ) then ../raw1 else ../raw2 }'/>
Now, given the statement below:
SMH: There is no way to make a choice branch fail when unparsing. (The only 'backtracking' when unparsing a choice is when the infoset contains no branch at all then the spec states that each branch is examined in turn until one is found that successfully applies defaults. But that's not really backtracking, as you can statically deduce the branch from the schema alone, so the 'default' branch to use can be computed up front).
So is there any way to fix this and make the above choice work?
This is the only thing I can think of, which ends up with entirely separate elements for parsing and unparsing.
I am not sure this works or should work.
<group name="hidG1"> <sequence> <!-- raw1 won't exist on unparsing --> <element name="raw1" type="xs:string" dfdl:lengthKind='pattern' dfdl:lengthPattern="[^\x7F]{0,99}" dfdl:terminator="%DEL;" minOccurs='0' dfdl:occursCountKind='implicit'/> <!-- raw2 will only occur parsing if raw1 doesn't and doesn't exist on unparsing --> <element name="raw2" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="100" minOccurs='0' dfdl:occursCountKind='expression' dfdl:occursCount="{ if (fn:exists(../raw1) then 0 else 1 }"/> <!-- the outputRaw will only be used when unparsing. On parsing it will just get "" as value, and nobody uses the value. --> <element name="outputRaw" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="{ 0 }" dfdl:outputValueCalc="{ if (fn:string-length(../value) eq 100 then ../value else fn:concat(../value, '') }" /> <sequence> </group>
<!-- to use, we hide the cruft in a hidden group --> <sequence dfdl:hiddenGroupRef="tns:hidG1"/> <element name='value' type='xs:string' dfdl:inputValueCalc='{ if (fn:exists( ../raw1 ) then ../raw1 else ../raw2 }'/>
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
I don't think it will work. Firstly A has minOccurs="0" which means there is the potential for it not to appear in the data, which will cause the variable to be set the wrong way. Secondly the dfdl:length expression for B is forward referencing to C which is not allowed. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org> Date: 14/07/2014 21:31 Subject: Re: Unparsing and choices - (related - Re: [DFDL-WG] Fw: Action 233 (deferred) - "byte order not sufficient..." - draft document on experience with binary format MIL-STD-2045 We'll I am a bit concerned if we cannot generally implement an asymetric parser and unparser inside the same schema. That is, if you have calculated element named C, then I think one should be able to have hidden element A which is used only when parsing (C computed from A), and hidden element B which is used only when unparsing, (B computed from C), where B is non-existing on parse, and A is non-existing on unparse. I think this trick may work: <!-- hidden. Found when parsing, not in infoset when unparsing --> <element name="A" type="xs:string" minOccurs="0" dfdl:occursCountKind= ..whatever.. /> <!-- boolean. Tells whether we're parsing or unparsing based on whether A is in infoset --> <dfdl:newVariableInstance ref="areParsing" value ="{fn:exists(../A)}"/> <!-- hidden. Length 0 when parsing. Unparsing gets length and value from C --> <element name="B" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="{ if ($areParsing) then 0 else fn:string-length(../C) }" default="" dfdl:outputValueCalc="{ ../C }" /> <!-- not hidden --> <element name="C" type="xs:string" dfdl:inputValueCalc="{ ../A }" /> On Mon, Jul 14, 2014 at 3:23 PM, Steve Hanson <smh@uk.ibm.com> wrote: The proposal for action 261 is that the dfdl:length expression is evaluated, so that will stop what you propose from working. For the MIL-STD-2045 'fixedOrTerminated' case then the sequence suggestion in the other email is much simpler. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org> Date: 14/07/2014 19:51 Subject: Re: Unparsing and choices - (related - Re: [DFDL-WG] Fw: Action 233 (deferred) - "byte order not sufficient..." - draft document on experience with binary format MIL-STD-2045 There has to be a way to get zero-length on parsing, but an outputValueCalc string on unparsing. <defineVariable name="dummyLength" defaultValue="0" external='true'/> <element name="outputRaw" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="{ $tns:dummyLength }" dfdl:outputValueCalc="{.....}" /> Would this work? The length expression isn't used when unparsing I believe in that some other elements are supposed to be storing the length information. That's the case here though the length value information isn't being stored, what is being stored is a way to determine the length. Maybe there's something simpler? On Mon, Jul 14, 2014 at 1:42 PM, Steve Hanson <smh@uk.ibm.com> wrote: Mike Section 17 of the spec says "It is a schema definition error if dfdl:inputValueCalc and dfdl:outputValueCalc are specified on the same element. " So outputRaw will give an SDE. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org> Date: 14/07/2014 16:43 Subject: Unparsing and choices - (related - Re: [DFDL-WG] Fw: Action 233 (deferred) - "byte order not sufficient..." - draft document on experience with binary format MIL-STD-2045 Breaking out this one issue for a separate discussion thread. Consider the choice below. There is a choice of two different elements named raw1 and raw2, then a value element which is computed based on one or the other of the alternatives. <choice> <!-- length kind pattern is needed to bound length to max of 99 --> <element name="raw1" type="xs:string" dfdl:lengthKind='pattern' dfdl:lengthPattern="[^\x7F]{0,99}" dfdl:terminator="%DEL;"/> <element name="raw2" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="100"/> </choice> <element name='value' type='xs:string' dfdl:inputValueCalc='{ if (fn:exists( ../raw1 ) then ../raw1 else ../raw2 }'/> Now, given the statement below: SMH: There is no way to make a choice branch fail when unparsing. (The only 'backtracking' when unparsing a choice is when the infoset contains no branch at all then the spec states that each branch is examined in turn until one is found that successfully applies defaults. But that's not really backtracking, as you can statically deduce the branch from the schema alone, so the 'default' branch to use can be computed up front). So is there any way to fix this and make the above choice work? This is the only thing I can think of, which ends up with entirely separate elements for parsing and unparsing. I am not sure this works or should work. <group name="hidG1"> <sequence> <!-- raw1 won't exist on unparsing --> <element name="raw1" type="xs:string" dfdl:lengthKind='pattern' dfdl:lengthPattern="[^\x7F]{0,99}" dfdl:terminator="%DEL;" minOccurs='0' dfdl:occursCountKind='implicit'/> <!-- raw2 will only occur parsing if raw1 doesn't and doesn't exist on unparsing --> <element name="raw2" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="100" minOccurs='0' dfdl:occursCountKind='expression' dfdl:occursCount="{ if (fn:exists(../raw1) then 0 else 1 }"/> <!-- the outputRaw will only be used when unparsing. On parsing it will just get "" as value, and nobody uses the value. --> <element name="outputRaw" type="xs:string" dfdl:lengthKind="explicit" dfdl:length="{ 0 }" dfdl:outputValueCalc="{ if (fn:string-length(../value) eq 100 then ../value else fn:concat(../value, '') }" /> <sequence> </group> <!-- to use, we hide the cruft in a hidden group --> <sequence dfdl:hiddenGroupRef="tns:hidG1"/> <element name='value' type='xs:string' dfdl:inputValueCalc='{ if (fn:exists( ../raw1 ) then ../raw1 else ../raw2 }'/> Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (2)
-
Mike Beckerle -
Steve Hanson