IBM 4690 data format

IBM 4690 is a widely used retail Point-of-Sale system. The transactions logs emitted by 4690 are in a compact, ASCII delimited binary format, known as TLOG. Each TLOG consists of a number of records, called 'strings'. Each 'string' consists of fields delimited by a colon (x2C), although some strings use double quote/colon/double quote instead (x22x2Cx22). The fields themselves can be fixed length or variable length but the delimiter is always present. The fields can have a number of physical reps but the most frequent rep is a packed number rep that is not BCD or S/390 packed decimal, and so DFDL can't model it. It is used to represent the vast majority of decimal and integer numbers in TLOG data.

This packed decimal format is best described by the operations that act on the data:

Pack: Take an ASCII string of characters, drop the first nibble of each byte, shuffle the remaining nibbles to the right, and if there is an odd number of nibbles pad with an xF nibble on the left.
Unpack: Discard any leading xF nibble, and expand each nibble into a byte by adding a x3 nibble in front, giving an ASCII string.

It can be seen that while the pack operation may be applied to any ASCII string, the unpack operation will only round trip if the original characters are in the range x30 to x3E, hence the operations were only intended to be used for packing and unpacking unsigned numbers. A convention has arisen however whereby the minus sign x2D may be used to indicate negative values, even though xD unpacks as x3D which is the equals sign. In summary:

Nibbles represent digits 0 - 9 in the usual BCD manner
A positive value is simply indicated by digits
A negative number is indicated by digits with the leading nibble being xD
If a positive or negative value packs to an odd number of nibbles, pad on the left with xF

Some examples help:

12 x12
123 xF1x23
-12 xFDx12
-123 xD1x23

Note that x00xD1x23 is not valid as this would unpack to 0=123 (ie, 0-123). The only nibble that can appear to the left of an xD nibble is an xF pad nibble. Or put another way, you can't blindly byte pad with x00 to make up a fixed length. If such byte padding is required, replace any xF and xD nibbles with x0 nibble, then byte pad with x00, then reinstate any xD nibble in the new leftmost nibble.

Proposal to add support to DFDL:

Existing dfdl:binaryNumberRep property gets a new enum 'ibm4690Packed'. It applies to all signed and unsigned simple types derived from xsd:decimal (both built-in and user-defined). The following DFDL properties are used in conjunction:

dfdl:byteOrder (although in practice IBM 4690 is always big-endian)
dfdl:binaryDecimalVirtualPoint

Note that dfdl:binaryPackedSignCodes does not apply - the only negative nibble is xD - and dfdl:binaryNumberCheckPolicy has no effect.

Where the DFDL specification provides for general behaviours for 'packed' and 'bcd', those behaviours apply also to 'ibm4690Packed'. Specifically:

The same dfdl:lengthKind enums and rules apply.
No rounding when serializing, a value that can't be accommodated is a processing error.
If the logical type is unsigned and a negative value is received, it is a processing error.

The same should be done for dfdl:binaryCalendarRep to allow TLOG packed decimal representation for calendars.

Regards

Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU