DFDL WG agreed to add the new enum to binaryNumberRep
and binaryCalendarRep. Errata will be taken.
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
----- Forwarded by Steve
Hanson/UK/IBM on 18/09/2012 16:43 -----
From:
Steve Hanson/UK/IBM
To:
dfdl-wg@ogf.org
Date:
17/09/2012 15:23
Subject:
Transaction
log data types from IBM 4690 point-of-sale terminals - add to DFDL spec?
IBM 4690 data format
IBM 4690 is a widely used retail Point-of-Sale
system. The transactions logs emitted by 4690 are in a compact, ASCII
delimited binary format, known as TLOG. Each TLOG consists of a number
of records, called 'strings'. Each 'string' consists of fields delimited
by a colon (x2C), although some strings use double quote/colon/double quote
instead (x22x2Cx22). The fields themselves can be fixed length or
variable length but the delimiter is always present. The fields can have
a number of physical reps but the most frequent rep is a packed number
rep that is not BCD or S/390 packed decimal, and so DFDL can't model it.
It is used to represent the vast majority of decimal and integer numbers
in TLOG data.
This packed decimal format is best described
by the operations that act on the data:
- Pack: Take an ASCII string of characters,
drop the first nibble of each byte, shuffle the remaining nibbles to the
right, and if there is an odd number of nibbles pad with an xF nibble on
the left.
- Unpack: Discard any leading xF nibble,
and expand each nibble into a byte by adding a x3 nibble in front, giving
an ASCII string.
It can be seen
that while the pack operation may be applied to any ASCII string, the unpack
operation will only round trip if the original characters are in the range
x30 to x3E, hence the operations were only intended to be used for packing
and unpacking unsigned numbers. A convention has arisen however whereby
the minus sign x2D may be used to indicate negative values, even though
xD unpacks as x3D which is the equals sign. In summary:
- Nibbles represent digits 0 - 9 in the
usual BCD manner
- A positive value is simply indicated
by digits
- A negative number is indicated by digits
with the leading nibble being xD
- If a positive or negative value packs
to an odd number of nibbles, pad on the left with xF
Some
examples help:
- 12
x12
- 123
xF1x23
- -12
xFDx12
- -123
xD1x23
Note that x00xD1x23 is
not valid as this would unpack to 0=123 (ie, 0-123). The only nibble that
can appear to the left of an xD nibble is an xF pad nibble. Or put another
way, you can't blindly byte pad with x00 to make up a fixed length. If
such byte padding is required, replace any xF and xD nibbles with x0 nibble,
then byte pad with x00, then reinstate any xD nibble in the new leftmost
nibble.
Proposal to add support to DFDL:
Existing dfdl:binaryNumberRep property
gets a new enum 'ibm4690Packed'. It applies to all signed and unsigned
simple types derived from xsd:decimal (both built-in and user-defined).
The following DFDL properties are used in conjunction:
- dfdl:byteOrder (although in practice
IBM 4690 is always big-endian)
- dfdl:binaryDecimalVirtualPoint
Note
that dfdl:binaryPackedSignCodes does not apply - the only negative nibble
is xD - and dfdl:binaryNumberCheckPolicy has no effect.
Where the DFDL specification provides
for general behaviours for 'packed' and 'bcd', those behaviours apply also
to 'ibm4690Packed'. Specifically:
- The same dfdl:lengthKind enums and rules
apply.
- No rounding when serializing, a value
that can't be accommodated is a processing error.
- If the logical type is unsigned and
a negative value is received, it is a processing error.
The
same should be done for dfdl:binaryCalendarRep to allow TLOG packed decimal
representation for calendars.
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU