- dfdl-wg - lists.ogf.org

Cancelled: OGF DFDL Working Group weekly call
by Steve Hanson 31 May '12

31 May '12

Public holiday in UK

1 0

Possible DFDL spec errata
by Steve Hanson 29 May '12

29 May '12

These were on the agenda for today's cancelled call. 2. Is parent/child alignment rule too strict? Spec property description for alignment says "The alignment of a child component must be less than or equal the alignment of its parent element, sequence or choice". Experiments with creating DFDL schema from C structures have shown that this rule is violated. It seems unnecessary. Specifically, C ensures that the alignment of objects in a repeating structure is preserved by rounding up the length of the structure, rather than aligning it. 3. Does DFDL need to support signed integer types with lengthUnits 'bits'? Spec only allows unsigned integers to have lengthUnits 'bits', but it is possible in C structures to delare bit fields for signed integers. They behave like two's complement. It looks like the spec is being too restrictive in its types for 'bits'. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 4

OGF DFDL WG Call 2012-05-29 - cancelled
by Steve Hanson 29 May '12

29 May '12

Not enough folk to make a quorum. Regards Steve Hanson Architect, DFDL, IBM SWG Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Agenda 2012-05-29
by Steve Hanson 29 May '12

29 May '12

Please find agenda for the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] Action 174: Making DFDL implementations easier
by Steve Hanson 25 May '12

25 May '12

Tim I tend to agree with endOfParent as optional. Expression language was kept in the core to handle occursCountKind 'expression' which is also in the core. The examples of binary data we have seen from NSA and ESA both have occurs counts in the data, in the same way as COBOL does. When you have untagged binary data, it's typically the way to provide an array size. It was felt that dropping this from the core did left us with too little capability. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM To: Steve Hanson/UK/IBM@IBMGB Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Date: 25/05/2012 10:07 Subject: Re: [DFDL-WG] Action 174: Making DFDL implementations easier 1) I would make endOfParent an optional feature - there are not many formats that require it. 3) There are many formats that do not require the expression language - it is only required when a property value or an assert/discriminator needs to query already-parsed data. On that basis, I think the entire expression language feature should be optional. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 From: Steve Hanson/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org Date: 25/05/2012 00:08 Subject: [DFDL-WG] Action 174: Making DFDL implementations easier Sent by: dfdl-wg-bounces(a)ogf.org Agreed on list, just need to answer questions 1) and 3) below. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 23/05/2012 18:21 ----- From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org Date: 15/05/2012 09:55 Subject: Making DFDL implementations easier Please see below for a proposal to make an additional set of DFDL features optional. The goal is to make it considerably easier to create a minimal conforming DFDL processor for binary data. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 11/05/2012 12:50 ----- Feature Detection Text representation for types other than String dfdl:representation="text" for Number, Calendar or Boolean types Delimiters dfdl:separator <> "" or dfdl:initiator <> "" or dfdl:terminator <> "" or dfdl:lengthKind="delimited" BCD calendars dfdl:binaryCalendarRep="bcd" Multiple schemas xs:include or xs:import in xsd Named Formats dfdl:defineFormat or dfdl:ref Choices xs:choice in xsd ** Arrays where size not known in advance dfdl:occursCountKind 'implicit', 'parsed', 'stopValue' ** Advanced expressions Advanced features of the DFDL expression language (tbd) ** Including one of these features mean that speculative parsing is needed. Remaining questions: 1) What about lengthKind 'endOfParent' ? 2) Is leaving out choices too restrictive? 3) Expression language subset The result is that a minimal conformant DFDL implementation just needs to support the following annotations and properties, and does not need speculative parsing. dfdl:element dfdl:sequence dfdl:format byteOrder encoding utf16width alignment alignmentUnits (bytes) fillByte leadingSkip trailingSkip lengthKind (explicit, implicit) length lengthUnits (bytes, characters) representation (binary) textPadKind textTrimKind textStringJustification textStringPadCharacter truncateSpecifiedLengthString decimalSigned binaryNumberRep binaryVirtualDecimalPoint binaryFloatRep (ieee) binaryBooleanTrueRep binaryBooleanFalseRep binaryCalendarRep (binarySeconds, binaryMilliseconds) binaryCalendarEpoch sequenceKind (ordered) occursCountKind (fixed, expression) occursCount Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

3 2

Action 174: Making DFDL implementations easier
by Steve Hanson 25 May '12

25 May '12

Agreed on list, just need to answer questions 1) and 3) below. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 23/05/2012 18:21 ----- From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org Date: 15/05/2012 09:55 Subject: Making DFDL implementations easier Please see below for a proposal to make an additional set of DFDL features optional. The goal is to make it considerably easier to create a minimal conforming DFDL processor for binary data. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 11/05/2012 12:50 ----- Feature Detection Text representation for types other than String dfdl:representation="text" for Number, Calendar or Boolean types Delimiters dfdl:separator <> "" or dfdl:initiator <> "" or dfdl:terminator <> "" or dfdl:lengthKind="delimited" BCD calendars dfdl:binaryCalendarRep="bcd" Multiple schemas xs:include or xs:import in xsd Named Formats dfdl:defineFormat or dfdl:ref Choices xs:choice in xsd ** Arrays where size not known in advance dfdl:occursCountKind 'implicit', 'parsed', 'stopValue' ** Advanced expressions Advanced features of the DFDL expression language (tbd) ** Including one of these features mean that speculative parsing is needed. Remaining questions: 1) What about lengthKind 'endOfParent' ? 2) Is leaving out choices too restrictive? 3) Expression language subset The result is that a minimal conformant DFDL implementation just needs to support the following annotations and properties, and does not need speculative parsing. dfdl:element dfdl:sequence dfdl:format byteOrder encoding utf16width alignment alignmentUnits (bytes) fillByte leadingSkip trailingSkip lengthKind (explicit, implicit) length lengthUnits (bytes, characters) representation (binary) textPadKind textTrimKind textStringJustification textStringPadCharacter truncateSpecifiedLengthString decimalSigned binaryNumberRep binaryVirtualDecimalPoint binaryFloatRep (ieee) binaryBooleanTrueRep binaryBooleanFalseRep binaryCalendarRep (binarySeconds, binaryMilliseconds) binaryCalendarEpoch sequenceKind (ordered) occursCountKind (fixed, expression) occursCount Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

3 2

OGF DFDL WG Call Minutes 2012-05-23
by Steve Hanson 23 May '12

23 May '12

Please find minutes from the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… 4 new actions to progress for next call. Regards Steve Hanson Architect, DFDL, IBM SWG Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Fw: String literal syntax for hexBinary ?? - dfdl:property semantic
by Steve Hanson 23 May '12

23 May '12

Agreed to drop dfdl:property() from 1.0. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 23/05/2012 18:19 ----- From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB Cc: dfdl-wg(a)ogf.org, Tim Kimber/UK/IBM@IBMGB Date: 19/04/2012 23:35 Subject: Re: [DFDL-WG] String literal syntax for hexBinary ?? - dfdl:property semantic ... But are we all happy that the result of dfdl:property when the property value is an expression, is the result of executing that expression? How complicated is that going to make implementations? Well, what is the use case for dfdl:property() at all? One example would be where there is a choice, and different record formats as the alternatives of the choice depending on the length or number of occurrences of some element. So a discriminator expression would refer to dfdl:property("length",...) for example. However, I do believe we could drop dfdl:property(....), and just say "use variables if you want this behavior". E.g., if I have dfdl:length="{....}" then as the schema author I can always introduce a variable myLen, set its value to that of the expression, change to dfdl:length="{ $myLen }" and where I would have used dfdl:property("length", path), instead just reference the variable as $myLen. I will have to define the variable, and use newVariableInstance so that the variable goes into and out of scope appropriately just like a property value does, but this isn't horribly complex. ...mikeb Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Actiion 173: DFDL String Literals Analysis for review
by Steve Hanson 23 May '12

23 May '12

----- Forwarded by Steve Hanson/UK/IBM on 23/05/2012 18:10 ----- From: "Mike Beckerle (Google Docs)" <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB Cc: Tim Kimber/UK/IBM@IBMGB Date: 21/04/2012 17:40 Subject: String Literals Analysis (smh(a)uk.ibm.com) I've shared String Literals Analysis Message from mbeckerle.dfdl(a)gmail.com: Online editable spreadsheet of all the various string-literal situations. This is better than that UML mixin diagram. Click to open: String Literals Analysis Google Docs makes it easy to create, store and share online documents, spreadsheets and presentations. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Unparsing hexBinary data
by Steve Hanson 23 May '12

23 May '12

Agreed on call today to allow padding using fillByte. No trimming though! Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM To: Steve Hanson/UK/IBM@IBMGB Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Date: 21/05/2012 10:11 Subject: Re: [DFDL-WG] Unparsing hexBinary data I think automatic padding to the specified or minimum length is better than the alternative ( throwing a processing error ). Arguably, throwing a processing error is more flexible because it offers the client an option to reject too-short data. But a dfdl:assert could do the same in the ( probably rare ) cases where that is the requirement. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 From: Steve Hanson/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org Date: 20/05/2012 07:22 Subject: [DFDL-WG] Unparsing hexBinary data Sent by: dfdl-wg-bounces(a)ogf.org When unparsing xs:string data of specified length, then if the data is short then it is padded to the specified or minimum length (depends on dfdl:lengthKind). Whether to do this padding or not is controlled by the dfdl:textPadKind property, However we don't offer the same flexibility for xs:hexBinary. There's no controlling property, so we always output exactly what was in the infoset, and if specified length and the data is short it is a processing error. In the absence of a property to control whether to pad, would we better off always padding (using dfdl:fillByte) to specified or minimum length? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0