DFDL WG agreed to add the new enum to binaryNumberRep and binaryCalendarRep. Errata will be taken. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 18/09/2012 16:43 ----- From: Steve Hanson/UK/IBM To: dfdl-wg@ogf.org Date: 17/09/2012 15:23 Subject: Transaction log data types from IBM 4690 point-of-sale terminals - add to DFDL spec? IBM 4690 data format IBM 4690 is a widely used retail Point-of-Sale system. The transactions logs emitted by 4690 are in a compact, ASCII delimited binary format, known as TLOG. Each TLOG consists of a number of records, called 'strings'. Each 'string' consists of fields delimited by a colon (x2C), although some strings use double quote/colon/double quote instead (x22x2Cx22). The fields themselves can be fixed length or variable length but the delimiter is always present. The fields can have a number of physical reps but the most frequent rep is a packed number rep that is not BCD or S/390 packed decimal, and so DFDL can't model it. It is used to represent the vast majority of decimal and integer numbers in TLOG data. This packed decimal format is best described by the operations that act on the data: Pack: Take an ASCII string of characters, drop the first nibble of each byte, shuffle the remaining nibbles to the right, and if there is an odd number of nibbles pad with an xF nibble on the left. Unpack: Discard any leading xF nibble, and expand each nibble into a byte by adding a x3 nibble in front, giving an ASCII string. It can be seen that while the pack operation may be applied to any ASCII string, the unpack operation will only round trip if the original characters are in the range x30 to x3E, hence the operations were only intended to be used for packing and unpacking unsigned numbers. A convention has arisen however whereby the minus sign x2D may be used to indicate negative values, even though xD unpacks as x3D which is the equals sign. In summary: Nibbles represent digits 0 - 9 in the usual BCD manner A positive value is simply indicated by digits A negative number is indicated by digits with the leading nibble being xD If a positive or negative value packs to an odd number of nibbles, pad on the left with xF Some examples help: 12 x12 123 xF1x23 -12 xFDx12 -123 xD1x23 Note that x00xD1x23 is not valid as this would unpack to 0=123 (ie, 0-123). The only nibble that can appear to the left of an xD nibble is an xF pad nibble. Or put another way, you can't blindly byte pad with x00 to make up a fixed length. If such byte padding is required, replace any xF and xD nibbles with x0 nibble, then byte pad with x00, then reinstate any xD nibble in the new leftmost nibble. Proposal to add support to DFDL: Existing dfdl:binaryNumberRep property gets a new enum 'ibm4690Packed'. It applies to all signed and unsigned simple types derived from xsd:decimal (both built-in and user-defined). The following DFDL properties are used in conjunction: dfdl:byteOrder (although in practice IBM 4690 is always big-endian) dfdl:binaryDecimalVirtualPoint Note that dfdl:binaryPackedSignCodes does not apply - the only negative nibble is xD - and dfdl:binaryNumberCheckPolicy has no effect. Where the DFDL specification provides for general behaviours for 'packed' and 'bcd', those behaviours apply also to 'ibm4690Packed'. Specifically: The same dfdl:lengthKind enums and rules apply. No rounding when serializing, a value that can't be accommodated is a processing error. If the logical type is unsigned and a negative value is received, it is a processing error. The same should be done for dfdl:binaryCalendarRep to allow TLOG packed decimal representation for calendars. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU