Forwarding to DFDL WG mailing list in
anticipation of discussion on this week's call.
Regards
Steve Hanson
Strategy, Common Transformation & DFDL
Co-Chair, OGF DFDL WG
IBM SWG, Hursley, UK,
smh@uk.ibm.com,
tel +44-(0)1962-815848
----- Forwarded by Steve
Hanson/UK/IBM on 06/07/2010 10:46 -----
| From:
| Steve Hanson/UK/IBM
|
| To:
| Suman Kalia/Toronto/IBM@IBMCA
|
| Cc:
| Alan Powell/UK/IBM@IBMGB, Tim Kimber/UK/IBM@IBMGB
|
| Date:
| 05/07/2010 16:06
|
| Subject:
| Re: Action : 088 define semantics of
choiceKind 'fixedLength' |
We discussed this briefly on the last
DFDL OGF WG call. The use case we are trying to support is where
a programming language has reserved memory for the longest branch of the
choice.
I would love to be able to drop the
dfdl:choiceKind property and instead use the dfdl:length property of the
parent element of the choice, set from the TD size attribute. In
other words, we do nothing special for choices;
- on input we expect dfdl:length bytes
or characters, we parse the chosen branch, if it is short we then skip
to the end
- on output we serialise the chosen
branch, if it is short we then pad out to dfdl:length bytes or characters
using dfdl:fillByte.
But this has some limitations...
a) It only works with choices that are
the content model of a complex type
b) It forces the DFDL modeller to calculate
the length of the choice
While b) is not a problem in practice
because the importer sets it and not the user, unfortunately a) is, because
the importers today generate a xs:group ref to a global choice group, so
there is nowhere to carry the dfdl:length property. I think the alternatives
are:
i) Change the importer to add a parent
element for each group. Con: This changes the logical model by inserting
an extra level.
ii) Allow dfdl:length to be carried
on embedded xs:choice (along with dfdl:lengthUnits & dfdl:lengthKind).
iii) New dfdl:choiceLength property
to be carried on embedded xs:choice (always bytes).
iv) dfdl:choiceKind as today. Con: length
from TD model is lost and must be recalculated.
v) dfdl:choiceKind but with limitations
to make the calculation easier. Con: length from TD model is lost and must
be recalculated.
The DFDL bottom-up 'model the data'
principle would point at i) but there are compatibility issues with existing
uses of the importer. That leaves iii) and v) as main contenders. It seems
a real shame to throw away the length that the TD model has calculated,
so I think we should discuss iii).
Regards
Steve Hanson
Strategy, Common Transformation & DFDL
Co-Chair, OGF DFDL WG
IBM SWG, Hursley, UK,
smh@uk.ibm.com,
tel +44-(0)1962-815848
| From:
| Suman Kalia/Toronto/IBM@IBMCA
|
| To:
| Alan Powell/UK/IBM@IBMGB, Steve Hanson/UK/IBM@IBMGB,
Tim Kimber/UK/IBM@IBMGB
|
| Date:
| 12/05/2010 16:25
|
| Subject:
| Re: Action : 088 define semantics of
choiceKind 'fixedLength' |
See the updated contentSize()
function in the attached mxsd file. It takes the largest size value for
the choice branch; in this case element x of size 20.
01
A.
05
b pic
9(3).
05
c occurs
1
to
10
times
depending
on
b.
10
x pic
x(20).
10
y redefines
x.
20
d pic
x(5).
20
e pic
9(3).
<?xml version="1.0"
encoding="UTF-8"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:complexType name="A">
<xsd:sequence>
<xsd:element
name="b">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfSimpleRep accessor="readWrite" addrUnit="byte"
alignment="byte" attributeInBit="false" contentSize="3"
offset="0" signFormat="trailing" signed="false"
size="3" typeName="MRCWFExternalDecimalRep" width="3"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="xsd:short">
<xsd:minInclusive value="0"/>
<xsd:maxInclusive value="999"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element
maxOccurs="10" minOccurs="1" name="c" type="A_c">
<xsd:annotation>
<xsd:appinfo source="http://www.wsadie.com/appinfo">
<dependingOn>/0/A;XSDComplexTypeDefinition/XSDParticle/XSDModelGroup/XSDParticle/b;XSDElementDeclaration</dependingOn>
</xsd:appinfo>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1" repeatRef="/0/A;XSDComplexTypeDefinition/XSDParticle/XSDModelGroup/XSDParticle/b;XSDElementDeclaration">
<cwfAggrRep accessor="readWrite" attributeInBit="false"
contentSize="mpy(20,val(1,1))"
offset="3" size="200">
<ArrayTD_arrayDescr alignmentKind="byte"
lowerBound="1" stride="20" strideInBit="false"
upperBound="val(1,1)"/>
</cwfAggrRep>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="A_c">
<xsd:group
ref="a_c"/>
</xsd:complexType>
<xsd:group name="a_c">
<xsd:sequence>
<xsd:group
ref="RedefinedElement_a_c_x_y">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfAggrRep accessor="readWrite" attributeInBit="false"
contentSize="20" offset="3" size="20"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
</xsd:group>
</xsd:sequence>
</xsd:group>
<xsd:group name="RedefinedElement_a_c_x_y">
<xsd:choice>
<xsd:element
name="x">
<xsd:annotation>
<xsd:appinfo source="http://www.wsadie.com/appinfo">
<initialValue kind="SPACE"/>
</xsd:appinfo>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfSimpleRep accessor="readWrite" addrUnit="byte"
alignment="byte" attributeInBit="false" characterSize="1"
contentSize="20" lengthEncoding="fixedLength" offset="3"
paddingCharacter="SPACE" prefixLength="0" size="20"
typeName="MRCWFStringRep" width="20"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:maxLength value="20"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="y" type="a_c_y">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfAggrRep accessor="readWrite" attributeInBit="false"
contentSize="8" offset="3" size="8"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:choice>
</xsd:group>
<xsd:group name="a_c_y">
<xsd:sequence>
<xsd:element
name="d">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfSimpleRep accessor="readWrite" addrUnit="byte"
alignment="byte" attributeInBit="false" characterSize="1"
contentSize="5" lengthEncoding="fixedLength" offset="3"
paddingCharacter="SPACE" prefixLength="0" size="5"
typeName="MRCWFStringRep" width="5"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:maxLength value="5"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element
name="e">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfSimpleRep accessor="readWrite" addrUnit="byte"
alignment="byte" attributeInBit="false" contentSize="3"
offset="8" signFormat="trailing" signed="false"
size="3" typeName="MRCWFExternalDecimalRep" width="3"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="xsd:short">
<xsd:minInclusive value="0"/>
<xsd:maxInclusive value="999"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:sequence>
</xsd:group>
<xsd:complexType
name="a_c_y">
<xsd:group ref="a_c_y"/>
</xsd:complexType>
<xsd:element name="msg_A"
type="A">
<xsd:annotation>
<xsd:appinfo
source="WMQI_APPINFO">
<MRMessage
messageDefinition="/0/msg_A;XSDElementDeclaration$MRObject"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:schema>
Suman Kalia
IBM Toronto Lab
WebSphere Message Broker Toolkit Architect and Development Lead
WebSphere Business Integration Application Connectivity Tools
http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.html
Tel : 905-413-3923 T/L 969-3923
Fax : 905-413-4850 T/L 969-4850
Internet ID : kalia@ca.ibm.com
From:
Suman Kalia/Toronto/IBM
To:
Alan Powell/UK/IBM@IBMGB,
Steve Hanson/UK/IBM@IBMGB, Tim Kimber/UK/IBM@IBMGB
Date:
05/12/2010 10:22 AM
Subject:
Action : 088
define semantics of choiceKind 'fixedLength'
Attached is the sample
cobol copy book and the mxsd generated from it.. It contains the TD model
annotations that we don't use in WMB. The TD model annotations that are
of particular interest for this action is the contentSize which is an expression
that tells the compiler runtime how much memory is actually allocated for
the repeating element. The size attribute tells what is the maximum it
can have based on the definition in the copy book.
From DFDL perspective,
the calculated length (reference Tim's note) would be similar to the contentSize()
attribute..
01
A.
05
b pic
9(3).
05
c occurs
1
to
10
times
depending
on
b.
10
d pic
x(5).
10
e pic
9(3).
<?xml version="1.0"
encoding="UTF-8"?><xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:complexType name="A">
<xsd:sequence>
<xsd:element
name="b">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfSimpleRep accessor="readWrite" addrUnit="byte"
alignment="byte" attributeInBit="false" contentSize="3"
offset="0" signFormat="trailing" signed="false"
size="3" typeName="MRCWFExternalDecimalRep" width="3"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="xsd:short">
<xsd:minInclusive value="0"/>
<xsd:maxInclusive value="999"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element
maxOccurs="10" minOccurs="1" name="c" type="A_c">
<xsd:annotation>
<xsd:appinfo source="http://www.wsadie.com/appinfo">
<dependingOn>/0/A;XSDComplexTypeDefinition/XSDParticle/XSDModelGroup/XSDParticle/b;XSDElementDeclaration</dependingOn>
</xsd:appinfo>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1" repeatRef="/0/A;XSDComplexTypeDefinition/XSDParticle/XSDModelGroup/XSDParticle/b;XSDElementDeclaration">
<cwfAggrRep accessor="readWrite" attributeInBit="false"
contentSize="mpy(8,val(1,1))"
offset="3" size="80">
<ArrayTD_arrayDescr alignmentKind="byte"
lowerBound="1" stride="8" strideInBit="false"
upperBound="val(1,1)"/>
</cwfAggrRep>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
<xsd:complexType name="A_c">
<xsd:group
ref="a_c"/>
</xsd:complexType>
<xsd:group name="a_c">
<xsd:sequence>
<xsd:element
name="d">
<xsd:annotation>
<xsd:appinfo source="http://www.wsadie.com/appinfo">
<initialValue kind="SPACE"/>
</xsd:appinfo>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfSimpleRep accessor="readWrite" addrUnit="byte"
alignment="byte" attributeInBit="false" characterSize="1"
contentSize="5" lengthEncoding="fixedLength" offset="3"
paddingCharacter="SPACE" prefixLength="0" size="5"
typeName="MRCWFStringRep" width="5"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:maxLength value="5"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element
name="e">
<xsd:annotation>
<xsd:appinfo source="WMQI_APPINFO">
<cwfInclRep messageSetDefaultRep="Binary1">
<cwfSimpleRep accessor="readWrite" addrUnit="byte"
alignment="byte" attributeInBit="false" contentSize="3"
offset="8" signFormat="trailing" signed="false"
size="3" typeName="MRCWFExternalDecimalRep" width="3"/>
</cwfInclRep>
</xsd:appinfo>
</xsd:annotation>
<xsd:simpleType>
<xsd:restriction base="xsd:short">
<xsd:minInclusive value="0"/>
<xsd:maxInclusive value="999"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:sequence>
</xsd:group>
<xsd:element name="msg_A"
type="A">
<xsd:annotation>
<xsd:appinfo
source="WMQI_APPINFO">
<MRMessage
messageDefinition="/0/msg_A;XSDElementDeclaration$MRObject"/>
</xsd:appinfo>
</xsd:annotation>
</xsd:element>
</xsd:schema>
Suman Kalia
IBM Toronto Lab
WebSphere Message Broker Toolkit Architect and Development Lead
WebSphere Business Integration Application Connectivity Tools
http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.html
Tel : 905-413-3923 T/L 969-3923
Fax : 905-413-4850 T/L 969-4850
Internet ID : kalia@ca.ibm.com
More work to do on this, but no time before the meeting. Let's discuss
the rules below, and refine afterwards:
If dfdl:choiceKind='fixedLength' then there must be at least one branch
of the choice for which all of the content has a calculable length.
Definition : A DFDL region is a region of the data stream described
by the DFDL grammar.
Note : we need to define this term - we use the term 'region' without
definition throughout the specification. Sometimes we call it a 'grammar
region'.
Definition: A DFDL region has a calculable length if the
length of its representation in units of bytes can be calculated from the
DFDL description alone.
The leading alignment region has a calculable length if the alignment of
the component is the same as the alignment of the parent.
The trailing alignment region has a calculable length if the alignment
of the component is the same as the alignment of the parent.
Q: Is this rule necessary?
The initiator region has a calculable length if all of the possible initiator
values have a calculable length.
An initiator value is a DFDL string literal. It has a calculable length
if the DFDL string literal does not contain any variable-length parts.
The terminator region has a calculable length if all of the possible terminator
values have a calculable length.
A terminator value is a DFDL string literal. It has a calculable length
if the DFDL string literal does not contain any variable-length parts.
A variable-length part is any of the following
- a character string in a variable-width encoding
- a DFLD entity in a variable-width encoding
- a DFDL generic entity which can match a variable number of characters
( NL, WSP+, WSP* )
The simple content region has a calculable length if both of the following
are true:
- either the encoding is a fixed-width encoding or the representation does
not contain characters.
- either lengthKind is 'explicit' and the length is not a DFDL expression
or lengthKind is 'implicit' and there is an implicit length for the simple
type and its dfdl:representation
The complex content region has a calculable length if all of the following
are true:
- either the encoding is a fixed-width encoding or the representation (
recursively ) does not contain characters.
- the element has lengthKind='explicit' and dfdl:length is not a DFDL expression
- all group members, recursively, are required ( minOccurs=maxOccurs )
regards,
Tim Kimber, Common Transformation Team,
Hursley, UK
Internet: kimbert@uk.ibm.com
Tel. 01962-816742
Internal tel. 246742
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU