Two corrections:
a) Use correct prefix for XPath integer()
function
b) Can't use dfdl:outputValueCalc (or
any other property) with dfdl:inputValueCalc
Regards
Steve Hanson
Strategy, Common Transformation & DFDL
Co-Chair, OGF DFDL WG
IBM SWG, Hursley, UK,
smh@uk.ibm.com,
tel +44-(0)1962-815848
----- Forwarded by Steve
Hanson/UK/IBM on 20/09/2010 17:29 -----
| From:
| Alan Powell/UK/IBM
|
| To:
| Steve Hanson/UK/IBM@IBMGB
|
| Date:
| 20/09/2010 15:57
|
| Subject:
| Re: Fw: DFDL functions testOn/Off and
setOn/Off - action 107 |
Steve
The spec doesn't allow inputValueCalc
and outputValueCalc on the same element.
|
|
Regards
|
|
|
| Alan Powell
|
|
|
| Development - MQSeries, Message Broker,
ESB
|
| IBM Software Group, Application and
Integration Middleware Software
|
| -------------------------------------------------------------------------------------------------------------------------------------------
|
| IBM
|
| MP211, Hursley Park
|
| Hursley, SO21 2JN
|
| United Kingdom
|
| Phone: +44-1962-815073
|
| e-mail: alan_powell@uk.ibm.com |
----- Forwarded by Steve
Hanson/UK/IBM on 20/09/2010 11:19 -----
| From:
| Steve Hanson/UK/IBM
|
| To:
| dfdl-wg@ogf.org
|
| Date:
| 20/09/2010 11:18
|
| Subject:
| DFDL functions testOn/Off and setOn/Off
- action 107 |
The expression language section of the
spec (21.5.3). defines some functions that make it easier to test/set individual
bits in a byte.
Action 107 was raised to question whether
they were needed and if so to complete the definitions. In particular the
spec for setOn/setOff seems odd.
Here's an example of where they are
useful, although upon investigation the functions need changing a bit.
A financial standard exists called ISO8583.
It is widely used with debit/credit card transactions. An ISO8583 data
stream consists of 128 optional fixed length fields with no tags or delimiters,
preceded by a mandatory bitmap that indicates the presence of all the other
fields.
In DFDL a nice approach is to model
each optional field as having dfdl:occursCountKind="expression"
with dfdl:occursCount referencing the corresponding bit in the bitmap and
casting its value to an integer to give 0 or 1. Each bit in the bitmap
would have its output value set using dfdl:outputValueCalc, true if the
corresponding field existed in the infoset.
The representation of the bitmap is
the interesting part. It can either be packed or unpacked.
Packed
In packed format the bitmap is a real
array of bits (8 bytes worth) which must be modelled as individual booleans
(ie, not an array) because each needs to carry a distinct dfdl:outputValueCalc
expression, as shown below. For example:
<complexType name="ISO8583_Packed">
<sequence>
<element name="BitMap">
<complexType>
<sequence>
<!-- each bit in the bitmap modelled individually
-->
<element name="Bit001" type="boolean"
dfdl:representation="binary"
dfdl:binaryBooleanTrueRep="1" dfdl:binaryBooleanFalseRep="0"
dfdl:lengthKind="explicit"
dfdl:length="1" dfdl:lengthUnits="bits"
dfdl:outputValueCalc="{if
fn:exists(../Field001) then true else false}"
/>
...
<element name="Bit064" type="boolean"
dfdl:representation="binary"
dfdl:binaryBooleanTrueRep="1" dfdl:binaryBooleanFalseRep="0"
dfdl:lengthKind="explicit"
dfdl:length="1" dfdl:lengthUnits="bits"
dfdl:outputValueCalc="{if
fn:exists(../Field064) then true else false}"
/>
</sequence>
</complexType>
</element>
<!-- each field
below determines its optional occurrence based on a bit in the bitmap field
-->
<element name="Field001"
type="string" minOccurs="0"
dfdl:occursKind="expression" dfdl:occurs="{xs:integer(../BitMap/Bit001)}"
/>
...
<element name="Field064"
type="string" minOccurs="0"
dfdl:occursKind="expression" dfdl:occurs="{xs:integer(../BitMap/Bit064)}"
/>
</sequence>
</complexType>
Unpacked
However, in the unpacked format the
value of each nibble of the bitmap is taken and unpacked as text (ASCII
or EBCDIC). Eg, bitmap '0000000100100011010001010110011110001001101010111100110111101111'
(64 bits) = x01x23x45x67x89xABxCDxEF (8 bytes) = x30x31x32x33x34x35x36x37x38x39x41x42x43x44x45x46
(16 bytes) in ASCII. This can be modelled as an unsigned long with
representation as a base 16 standard text number.
The problem is then how to test and
set the individual bits in the unsigned long. That's where the DFDL functions
come in useful, because there are no XPath functions for bitwise operations.
If the functions are defined as follows:
xs:boolean dfdl:testBit(xs:unsignedLong,
xs:unsignedShort)
Returns Boolean true if the bit number given by arg #2 is set on
in the integer given by arg #1, otherwise returns Boolean false.
xs:unsignedLong dfdl:setBits(xs:boolean+)
Returns an integer being the value of the bit positions provided
by the Boolean arguments, where true=1, false=0. The # of args must be
a multiple of 8.
Note that the bit numbering
goes from left to right, in accordance with section 12.3.7.2 of the spec.
Applying this to the example above gives:
<complexType name="ISO8583_Unpacked">
<sequence>
<element name="BitMapInt"
type="unsignedLong"
dfdl:representation="text" dfdl:textNumberRep="standard"
dfdl:textNumberBase="16" dfdl:encoding="ascii"
dfdl:lengthKind="explicit" dfdl:length="16"
dfdl:lengthUnits="characters"
dfdl:outputValueCalc="{dfdl:setBits(fn:exists(../Field001),
...
fn:exists(../Field064)}"
/>
<!-- each field
below determines its optional occurrence based on a bit in the bitmap field
-->
<element name="Field001"
type="string" minOccurs="0"
dfdl:occursKind="expression" dfdl:occurs="{xs:integer(dfdl:testBit(../BitMapInt,
1)}" />
...
<element name="Field064"
type="string" minOccurs="0"
dfdl:occursKind="expression" dfdl:occurs="{xs:integer(dfdl:testBit(../BitMapInt,
64)}" />
</sequence>
</complexType>
To keep the unpacked model as close
to the packed model as possible, we can incorporate the logical BitMap
from the packed model and hide BitMapInt. This means the Fieldxxx
elements are the same for each.
<complexType name="ISO8583_Unpacked">
<sequence>
<sequence dfdl:hiddenGroupRef="HiddenBitMap"
/>
<element name="BitMap">
<complexType>
<sequence>
<!-- each bit in the bitmap modelled individually
but derived -->
<element name="Bit001" type="boolean"
dfdl:inputValueCalc="{dfdl:testBit(../BitMapInt,
1)}" />
...
<element name="Bit064" type="boolean"
dfdl:inputValueCalc="{dfdl:testBit(../BitMapInt,
64)}" />
</sequence>
</complexType>
</element>
<!-- each field
below determines its optional occurrence based on a bit in the bitmap field
-->
<element name="Field001"
type="string" minOccurs="0"
dfdl:occursKind="expression" dfdl:occurs="{xs:integer(../BitMap/Bit001)}"
/>
...
<element name="Field064"
type="string" minOccurs="0"
dfdl:occursKind="expression" dfdl:occurs="{xs:integer(../BitMap/Bit064)}"
/>
</sequence>
</complexType>
<group name="HiddenBitMap">
<sequence>
<element name="BitMapInt"
type="unsignedLong"
dfdl:representation="text" dfdl:textNumberRep="standard"
dfdl:textNumberBase="16" dfdl:encoding="ascii"
dfdl:lengthKind="explicit" dfdl:length="16"
dfdl:lengthUnits="characters"
dfdl:outputValueCalc="{dfdl:setBits(fn:exists(../Field001),
...
fn:exists(../Field064)}"
/>
</sequence>
</group>
For both packed and unpacked, the BitMap
element could be hidden as well, as there is no need for it in the infoset.
Regards
Steve Hanson
Strategy, Common Transformation & DFDL
Co-Chair, OGF DFDL WG
IBM SWG, Hursley, UK,
smh@uk.ibm.com,
tel +44-(0)1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU