- dfdl-wg - lists.ogf.org

DFDL: Minutes from OGF WG call, February-18-2009
by Alan Powell 19 Feb '09

19 Feb '09

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, February-18-2009, Attendees Alan Powell (IBM) Steve Hanson (IBM) Dave Glick (drac) Apologies Mike Beckerle(Oco) 1. XSD 1.1 Discussion deferred 2. Escape Schemes Further discussion of latest proposal. Noted the need to properly describe the function of escapeSceheme and in particular distinguish from substitution mechanisms such as XML entities which are sometimes called escapes. SH felt there could be confusion with the two use cases of escapeCharacter. Suggested separate properties. Action SH to document issues and improve wording Discussed whether comment type escapes should be supported. Decided they are not needed on V1. AP to confirm that if IBM products provide support. 2. Review Actions Action 20 - There had been a number of email exchanges following on from last weeks call. Decided to keep 'zoned' as it is well known to users who need the support. Discussion on whether dfdl should allow unsigned logical types to have 'signed' phsyical representation. If so what should be the behaviour when there is a mismatch of signs. Action: AP to describe behaviour 3. calendarFormats SH and AP have been discussing the behaviour of strict and lax parsing which needs a more complete description. 4. TypeSubsitution AP raised whether typeSubstitution needs to be supported in DFDL v1. 5. AOB DG to take a look at Work items 5 and 12 while continuing review MB will not be available next week Next call 25 February 14:00 Meeting closed, 15:00 GMT Actions raised at this meeting No Action Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 04/02: changed from locale to specific properties 18/2: Need more investigation of ICU strict/lax behaviour. 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 11/02: proposal largely agreed. SH to make minor changes 18/02: AP to document unsigned type behaviour 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 04/02 Call arranged for Friday 11/02: Call took place. Identified useful changes. Consolidate with previous list. 024 String XML type 025 Escape schemes 21/1: discussed requirements 04/02: AP/SH to describe behaviour for known length text fields. Need to discuss if comment escapes should be supported. 11/02 new draft distributed: 18:02: SH up document concerns 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) 031 DG: Review dfdl v033 11/02: Initial comments received 18/02: Will include work items 5 and 12. Closed actions: 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work 026 Remove duration complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 11 February 2009
by Alan Powell 12 Feb '09

12 Feb '09

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 14:00 GMT, 11 February 2009 Attendees Alan Powell (IBM) Steve Hanson (IBM) Dave Glick (drac) Aplogies Mike Beckerle(Oco) 1. XSD 1.1 Went through Schema 1.1 changes presentation from Sandy Gao. Some items need further investigation 2. Review Actions Action 20 - Discussed suggested changes from SH. Observed that non-overpunching zoned was the same as text. Need to decide if unsigned types are allowed to have representation zoned. Other actions updated below 3. Escape Schemes Discussed latest proposal. General agreement that list of references was better than list of pairs of delimiters. Observed that property names within xxxFormats don't need type prefix eg calendarPatternKind -> patternKind Need to discuss if comment escapes should be supported. 4. Calendar/Number formats Didn't discuss 4. AOB Next call 18 February 14:00 Meeting closed, 15:00 GMT Actions raised at this meeting No Action Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 04/02: changed from locale to specific properies 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 11/02: proposal largely agreed. SH to make minor changes 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 04/02 Call arranged for Friday 11/02: Call took place. Idnetified useful changes. Consolidate with previous list. 024 String XML type 025 Escape schemes 21/1: discussed requirements 04/02: AP/SH to describe behaviour for known length text fields. Need to discuss if comment escapes should be supported. 11/02 new draft distributed: 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) 031 DG: Review dfdl v033 11/02: Initial comments received Closed actions: 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work 026 Remove duration complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: DFDL OGF WG call - Action 020
by Steve Hanson 12 Feb '09

12 Feb '09

It was noted on the call this week that there is an alternative to my zoned decimal overpunching proposal i) below. I said: - If it is an unsigned type then DFDL expects the rightmost byte to have a zone nibble when parsing, and outputs a zone nibble when unparsing. - If it is a signed type then DFDL expects it to have a sign nibble when parsing, and outputs a sign nibble when unparsing. But my unsigned type behaviour could be achieved by specifying a rep of text instead of zoned. If that is the case, the alternative is to only allow zoned rep for signed decimal logical types. Thoughts? Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 05/02/2009 10:17 ----- Steve Hanson/UK/IBM 28/01/2009 13:54 To DFDL Working Group cc Subject DFDL OGF WG call - Action 020 Action 020: 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position a) Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy Add new property to section 15.4 Properties Specific to Number with Binary representation. binaryNumberCheckPolicy Enum Values are ?strict? and ?lax?. Indicates how lenient to be when parsing binary numbers. If ?lax? then the parser tolerates all valid alternatives where such alternatives exist. Specifically, for binaryNumberRepresentation = 'packed' the sign nibble for positive, negative, unsigned and zero is allowed to be any of the valid respective values. On unparsing, the specified value is always used. Also suggest changing some of the other property names in 15.4: "decimalVirtualPoint" -> "binaryDecimalVirtualPoint" "packedDecimalSignCodes" -> "binaryPackedSignCodes" And changing binaryNumberRepresentation enumeration: "BCD" -> "bcd" b) Zoned decimals: How to decide to overpunch and sign position Spec assumes that overpunching of the rightmost character always takes place. IBM architecture allows no overpunching (ie, Fx instead of Cx/Dx) - this is supported by IBM MRM & WTX parsers. Additionally IBM MRM parser allows separate sign byte, and sign byte on left. Let's deal with these separately: i) No overpunching. The IBM architecture allows the rightmost byte to have a zone (Fx) or a sign (Cx/Dx) as the left nibble. I don't see why we can't base what to expect when parsing, and output when unparsing, on the logical xsd type. - If it is an unsigned type then DFDL expects the rightmost byte to have a zone nibble when parsing, and outputs a zone nibble when unparsing. - If it is a signed type then DFDL expects it to have a sign nibble when parsing, and outputs a sign nibble when unparsing. For analogy with DFDL packed decimals, it seems at first glance that we should also extend the numberCheckPolicy 'lax' setting to treat a zone nibble as a +ve sign nibble for a signed type. However, IBM iSeries always outputs Fx to mean +ve but accepts both Fx & Cx on input. It is perhaps better therefore that DFDL always tolerates Fx when parsing a signed zoned decimal, otherwise iSeries users would always have to set numberCheckPolicy to 'lax', which might have other implications in the future. ii) Separate sign byte. I don't believe the IBM architecture allows this. I'm not sure where MRM got this from. I don't think DFDL needs to support it. iii) Sign byte on left. I don't believe the IBM architecture allows this. I'm not sure where MRM got this from. I don't think DFDL needs to support it. Conclusion: No new DFDL properties needed, but words need adding to explain zoned parse/unparse behaviour better. Also suggest changing property names: "zonedDecimalSignStyle" -> "numberZonedSignStyle" "zeroNumberRep" -> "numberZeroRep" Should also make clear that any explicit negative pattern in numberPattern will be ignored if the xsd type is unsigned. (We could make this an error but it precludes creation of a textNumberFormat that works with both signed and unsigned types, plus pattern "##0.0" implictly is equivalent to "##0.0;(##0.0)" ). Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Alan Powell/UK/IBM@IBMGB Sent by: dfdl-wg-bounces(a)ogf.org 23/01/2009 13:36 To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] DFDL: Minutes from OGF WG call, 21 January 2009 Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 14:00 GMT, 21 January 2009 Attendees Alan Powell (IBM) Mike Beckerle(Oco) Apologies Steve Hanson (IBM) 1. XSD 1.1 Deferred to next call 2. Calendar formats Discussed updated (v4) supplement emailed by AP Agreed millisec/secSinceEpoc cannot be implied by length of logical data so need seperate enumerations. Observed that these options were really combination of 3 properties binary, length and sec/millisec. Suggested renaming to binarySeconds and binaryMilliseconds Packed calendars: decided that need to be able to specify at least the packedDecimalSignCodes property rather than assuming a default so reference will be added to calendar description Locale needs to be specified for numberformats and calendarFormats (didn't identify any other areas) as it modifies the behaviour of ICU. Decided to add locale to numberFormat and CalendarFormat 3. Escape Schemes Agreed need for multiple escape delimiter pairs but not nested. Need an escape for escape character even though in most cases this will be the same character, eg /n //, There are some formats that have a different escape, eg /n &/. Only need single escape characters and one level of escape characters. Discussed how to deal with comments of the form /* comment */ where the escape delimiters are also the initiator and terminator of the field. Semantic needed is 'only look for field terminator not any parent terminator or any other syntax elements'. May fall out naturally from the speculative parsing rules. Need further discussion. 4. AOB Next call 28 January 14:00 Meeting closed, 15:00 GMT Actions raised at this meeting No Action 031 Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 023 MB: Review Schema 1.1 024 String XML type 025 Escape schemes 21/1: discussed requirements 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) 030 AP: confirm with WTX that can drop duration 21/6: WTX confirm that they do not have a duration type so do not need it in dfdl. Will drop from spec. Closed Closed actions: 030 AP: confirm with WTX that can drop duration 21/6: WTX confirm that they do not have a duration type so do not need it in dfdl. Will drop from spec. Closed 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 3

DFDL: Agenda for OGF WG call, 11 February 2009
by Alan Powell 09 Feb '09

09 Feb '09

Suggested agenda 1. XML Schema 1.1 2. Review Actions. In particular action 20 and solution sent by SH 3. Escape schemes Review attached document. 4. Calendar Formats 5. Plan to finish specification 6. AOB Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 04 February 2009
by Alan Powell 05 Feb '09

05 Feb '09

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 14:00 GMT, 04 February 2009 Attendees Alan Powell (IBM) Mike Beckerle(Oco) Steve Hanson (IBM) Dave Glick (drac) 1. XSD 1.1 SH and AP have scheduled a call with Sandy for this Friday 2. Welcome David Glick MB welcomed David Glick to the WG. David gave some background to his interest in DFDL. He is currently leading a SBIR project for the US Navy which involves efficiently parsing binary network data. DG will review draft 033 and submit comments 3. Escape Schemes Further discussion of proposal distributed by AP, in particular scanning/skipping known length text fields. SH and AP pointed out that the (assumed) speculative parsing behaviour required that known length text fields needed to be scanned if terminating syntax was in scope so last week's proposal was not correct. AP/SH to write up description of speculative parsing, scanning, escaping text fields. Need to discuss if comment escapes should be supported. 4. Calendar/Number formats AP raised question about whether locale or specific properties should be used to define properties such as decimalseparator in numberFormat and langauge in calendarFormat. Decided that specific properties should be used as it gave more control and didn't require users to 'find' the correct locale for a format. 4. AOB Next call 11 February 14:00 Meeting closed, 15:00 GMT Actions raised at this meeting No Action 031 DG: Review dfdl v033 Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 04/02: changed from locale to specific properies 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 04/02 Call arranged for Friday 024 String XML type 025 Escape schemes 21/1: discussed requirements 04/02: AP/SH to describe behaviour for known length text fields. Need to discuss if comment escapes should be supported. 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) Closed actions: 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work 026 Remove duration complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 21 January 2009
by Alan Powell 30 Jan '09

30 Jan '09

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 14:00 GMT, 28 January 2009 Attendees Alan Powell (IBM) Mike Beckerle(Oco) Steve Hanson (IBM) 1. XSD 1.1 SH and AP will arrange a call with Sandy Gao 2. Escape Schemes Discussed proposal distributed by AP Escape schemes apply to all logical types with text representation As with other defnexxxFormats it is the escapeSchemeRefermce that is in scope not the properties within the escape scheme. SH proposed allowing multiple escape refs for each alternate escape delimiter pairs rather than defining them in on escape scheme. When escapeSchemeRef is blank then no escape scheme applies Discussion about whether escape schemes apply to known length as well as variable length text fields. Earlier decision to scan known length text fields for parent syntax items implies need to allow escape. Suggested syntax 1. escapeSchemeRef= blank means no escaping and don't scan 2. escapeSchemeRef=something but escape delimiters and escapecharacter are blank means no escape but scan for syntax 3. escapeSchemeRef=something but escape delimiters and escapeCharacter are something means escape and scan for syntax Subsequent discussions indicate that some products scan if there are syntax items in scope even if known length. Needs further discussion. Need to discuss if comment escapes should be supported. 4. AOB Next call 04 February 14:00 Meeting closed, 15:00 GMT Actions raised at this meeting No Action 031 Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 024 String XML type 025 Escape schemes 21/1: discussed requirements 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) 030 AP: confirm with WTX that can drop duration 21/6: WTX confirm that they do not have a duration type so do not need it in dfdl. Will drop from spec. Closed Closed actions: 030 AP: confirm with WTX that can drop duration 21/6: WTX confirm that they do not have a duration type so do not need it in dfdl. Will drop from spec. Closed 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Introduction and Participation
by Dave Glick 30 Jan '09

30 Jan '09

Hi, I would like to briefly introduce myself. I am currently leading a SBIR project for the US Navy (SBIR is a program designed to grow small businesses and emphasizes innovative research and application). A large part of what I'm working on involves the conversion of arbitrary binary data to XML for processing and back again. After evaluating several alternatives, I have come to believe that DFDL is pretty close to perfect for what we need. As there are no commercially available DFDL parsers available, part of my project will be to create such a parser (at least for the subset of the standard relevant to my effort - hopefully building to a complete implementation at some point). To that end, I will be approaching the standard from the perspective of an implementation developer and hope to be able to participate in the working group in whatever ways are useful. The level of my involvement depends on funding (relatively small right now), but I should know within a short timeframe (2 months or so) whether I will be funded for the next several years at a much higher level. In any case, I am already generating questions and comments for the most recently published draft (033). Would it be valuable for me to send them to someone? How else can I be of assistance and participate now and in the future? Thanks, Dave --- David Glick | dglick(a)dracorp.com <mailto:dglick@dracorp.com> | 703.299.0700 x212 Data Research and Analysis Corp. | www.dracorp.com <http://www.dracorp.com>

2 1

DFDL: Minutes from OGF WG call, 21 January 2009
by Alan Powell 28 Jan '09

28 Jan '09

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 14:00 GMT, 21 January 2009 Attendees Alan Powell (IBM) Mike Beckerle(Oco) Apologies Steve Hanson (IBM) 1. XSD 1.1 Deferred to next call 2. Calendar formats Discussed updated (v4) supplement emailed by AP Agreed millisec/secSinceEpoc cannot be implied by length of logical data so need seperate enumerations. Observed that these options were really combination of 3 properties binary, length and sec/millisec. Suggested renaming to binarySeconds and binaryMilliseconds Packed calendars: decided that need to be able to specify at least the packedDecimalSignCodes property rather than assuming a default so reference will be added to calendar description Locale needs to be specified for numberformats and calendarFormats (didn't identify any other areas) as it modifies the behaviour of ICU. Decided to add locale to numberFormat and CalendarFormat 3. Escape Schemes Agreed need for multiple escape delimiter pairs but not nested. Need an escape for escape character even though in most cases this will be the same character, eg /n //, There are some formats that have a different escape, eg /n &/. Only need single escape characters and one level of escape characters. Discussed how to deal with comments of the form /* comment */ where the escape delimiters are also the initiator and terminator of the field. Semantic needed is 'only look for field terminator not any parent terminator or any other syntax elements'. May fall out naturally from the speculative parsing rules. Need further discussion. 4. AOB Next call 28 January 14:00 Meeting closed, 15:00 GMT Actions raised at this meeting No Action 031 Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 023 MB: Review Schema 1.1 024 String XML type 025 Escape schemes 21/1: discussed requirements 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) 030 AP: confirm with WTX that can drop duration 21/6: WTX confirm that they do not have a duration type so do not need it in dfdl. Will drop from spec. Closed Closed actions: 030 AP: confirm with WTX that can drop duration 21/6: WTX confirm that they do not have a duration type so do not need it in dfdl. Will drop from spec. Closed 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

Escape Scheme propsal
by Alan Powell 26 Jan '09

26 Jan '09

All Attached is a proposal for specifying escape schemes Points of discussion As specified escaping applies only to String logical types represented as text. Is it needed for Numbers or Calendars types with text representation? I think not. Message Broker only escapes variable length text fields but I have not made that restriction. Is the interaction between escaping and padding clear enough? Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Fw: Thirdn pass at CalendarFormat
by Alan Powell 20 Jan '09

20 Jan '09

Updated proposal attached. Still need to decide how to specify locale for calendarformat and numberformat Main changes: dropped zoned (and therefore textCalendarRep) dropped binary for c-time structure Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: Alan Powell/UK/IBM To: Steve Hanson/UK/IBM@IBMGB, mbeckerle.dfdl(a)gmail.com Date: 18/12/2008 16:54 Subject: Fw: Second pass at CalendarFormat Guys I have added more info to the calendar supplement. see changes table. An open question is how to deal with locales. ICU by default uses the local locale which is probably not correct for dfdl. It is also not good that we a property with an enumeration of Monday. Tuesday etc. Also need to consider nulls, defaults and zeroRepresentation (added) I couldn't decide if it was better to have separate calendarformats for text and binary as we do for numberformats. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 ----- Forwarded by Alan Powell/UK/IBM on 18/12/2008 16:47 ----- From: Alan Powell/UK/IBM To: Steve Hanson/UK/IBM@IBMGB, mbeckerle.dfdl(a)gmail.com Date: 12/12/2008 16:42 Subject: First pass at CalendarFormat I have done an initial updated of the Data/time supplement following the numberFormat pattern. Comments [attachment "ggf-dfdl-supplement-date-time-properties-v1.0-002.doc" deleted by Alan Powell/UK/IBM] Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU #### ggf-dfdl-supplement-date-time-properties-v1.0-003.doc moved to MyAttachments Repository V3.8 () on 06 January 2009 by Alan Powell. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0