- dfdl-wg - lists.ogf.org

DFDL Example for 2D arrays
by Jyotishman Pathak 06 Jan '09

06 Jan '09

Hello all, I am a newbie in DFDL, and trying to grasp some of the key aspects going through the latest core spec: http://forge.gridforum.org/sf/go/doc15397?nav=1 Pardon me if this question is nonsensical, but I wonder if there are any examples of how a DFDL schema/document will look like if Matrix.data is representing a 2D array? Or if there are any sample DFDL documents for an image file (jpg, png etc.). The simple example that you have in pages 19--20 of the core spec is indeed helpful to understand the guts of the idea, but I would like to see somewhat more complex examples. I look forward to your help. Thanks, - Jyoti

2 1

Agenda for DFDL WG call - 17-12-2008
by Alan Powell 17 Dec '08

17 Dec '08

Suggested agenda 1. Moving to XSD 1.1 as base for DFDL 2. Padding to minlength Comments on last call discussion 3. Implicit Lengths Comments on last call discussion 4. Calendar format. review AP email 5. Review actions 6. AOB Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

Invitation: OGF DFDL Working Group weekly call (14 Jan 14:00 GMT in DE2K09)
by Steve Hanson 16 Dec '08

16 Dec '08

Calendar Entry ---------------------------------------------------------------------------------------------------------------------------- Subject Chair OGF DFDL Working Group weekly call Steve Hanson/UK/IBM Invitees Date Required (to) 14/01/2009 Alan Powell/UK/IBM, mbeckerle(a)oco-inc.com, Suman Kalia/Toronto/IBM@IBMCA Time 14:00 - 15:00 (1 hour) Optional (cc) dfdl-wg(a)ogf.org, Tim Kimber/UK/IBM Proposed start -1 at Proposed end -1 at Original start 14/01/2009 at 14:00 Original end 14/01/2009 at 15:00 Repeats First meeting 14/01/2009 Repeating Every week on Wednesday until 16/12/2009 Where Location DE2K09 Participant Passcode: 8190865 Country Toll Numbers Freephone/Toll Free Number AUSTRIA 43-1-928-9652 BELGIUM 32-2-400-6791 0800-4-9612 DENMARK 45-7014-0237 FRANCE LYON: 33-4-26-69-12-74 080-511-3725 FRANCE MARSEILLE: 33-4-86-06-00-74 080-511-3725 FRANCE PARIS: 33-1-70-70-81-58 080-511-3725 GERMANY 49-69-2222-20363 0800-100-6504 IRELAND 353-1-247-7768 ITALY 39-02-3600-3648 800-985-679 LUXEMBOURG 352-27-000-1312 NETHERLANDS 31-20-713-2741 0800-020-3237 SPAIN 34-91-414-1542 800-099-667 SWEDEN 46-8-566-10-780 SWITZERLAND 41-44-580-3334 0800-000-516 UNITED KINGDOM BIRMINGHAM: 44-121-210-9014 0800-279-9193 UNITED KINGDOM GLASGOW: 44-141-202-3214 0800-279-9193 UNITED KINGDOM LEEDS: 44-113-301-2114 0800-279-9193 UNITED KINGDOM LONDON: 44-20-7019-0808 0800-279-9193 UNITED KINGDOM MANCHESTER: 44-161-601-1414 0800-279-9193 USA 866-505-4412

1 0

DFDL: Minutes from OGF WG call, 10 December 2008
by Alan Powell 10 Dec '08

10 Dec '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 10 December 2008 Attendees Steve Hanson (IBM) Alan Powell (IBM) Appolgies Mike Beckerle(Oco) 1. Padding to minlength Discussed AP proposal and subsequent emails. SH comments "Alan - I see what you have done - you've added textStringPad but also made it the controlling property - ie, you look at textStringPad first. (This matches what IBM WTX does). That's ok - as long as you don't have textStringPad controlling trimming. So here's a proposal that reflects that. Note that I have made textPadKind applicable to strings, numbers and calendars (like textTrimKind) - I think it's the justification direction and the pad char that vary per type, not whether to pad or not. I've also dropped the stuff about a length prefixed string being trimmed to a max length - we can re-instate in the appropriate place if needed - was that to do with ensuring you couldn't pass a string whose length could not be accommodated in the prefix type (because that's not how it was worded)." Refined proposal Property Name Description textTrimKind Enum Valid values ?none?, ?padChar? Indicates whether to trim data on input. 'none': No trimming takes place 'padChar': If lengthKind results in a fixed length then the padding character can be trimmed instead, as controlled by textStringJustification. Annotation: dfdl:element (simple type ?string?) textPadKind Enum Valid values ?none?, ?fixed?, ?minimum' Indicates whether to pad data on output. ?none?: The string is expected to match the length, it is an error otherwise. 'fixed': If lengthKind results in a fixed length then the data can be padded to the fixed length, as controlled by textStringJustification. 'minimum': If lengthKind results in a fixed length then the data can be padded to the length given by dfdl:outputMinLength property, as controlled by textStringJustification. Property Name Description textStringJustification Enum Valid values ?left?, ?right?, ?center? Output: 'left': Justifies to the left and adds padding chars to the string contents if the string is too short, to the length determined by the textPadKind property. 'right': Justifies to the right and adds padding chars to the string contents if the string is too short, to the length determined by the textPadKind property. ?center?: Adds equal padding chars left and right of the string contents if the string is too short, to the length determined by the textPadKind property. It adds one extra padding char before if needed. Input: 'left': Trims any padding characters from the right of the string, according to textTrimKind property. 'right': Trims any padding characters from the left of the string, according to textTrimKind property. ?center? Trims any padding characters from the left and right of the string, according to textTrimKind property. [OMG/CAM property justification] Annotation: dfdl:element (simple type ?string?) Similar for textNumberJustification and textCalendarJustification 2. Implicit Lengths Decided to disallow zoned for Float and Double (MB to confirm) Use numberPattern for all zoned lengths instead of fixed length implied by type Simple type text (+ xml) binary String Length/maxlength n/a Float Text: numberPattern (length of positive subpattern) 32 Double Text: numberPattern 64 Decimal/Integer Text: numberPattern Zoned:numberPattern Pkd/bcd: (TotalDigits+1)/2 Long, UnsignedLong Text: numberPattern Zoned:numberPattern binary: 64 Pkd/bcd: (TotalDigits+1)/2 Int, Unsignedint Text: numberPattern Zoned:numberPattern binary: 32 Pkd/bcd: (TotalDigits+1)/2 Short, Unsignedshort Text: numberPattern Zoned:numberPattern binary: 16 Pkd/bcd: (TotalDigits+1)/2 Byte, Unsignedbyte Text: numberPattern Zoned:numberPattern binary: 8 Pkd/bcd: (TotalDigits+1)/2 DateTime Calendarpattern Date Calendarpattern Time Calendarpattern Duration Calendarpattern Boolean Length of textBooleanTrue 32 HexBinary n/a Length/maxlength 3. Zoned Decimal Discussed whether zonedDecimalSignStyle should be moved out of numberFormat. Noticed that no option for specifying overpunching or not. (added to action 020) Need more investigation of ICU behaviour. 4. AOB Next call 17 December Meeting closed, 13:00 GMT Actions raised at this meeting No Action Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 12/11: Updates have been sent to wtx to review. Will chase up for comments 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 26/11: discussed and AP to suggest dfdl:minlength 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 023 MB: Review Schema 1.1 024 String XML type 025 Escape schemes 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 complete 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules complete 014 (from F2F) Occurs, OccurSeparator changes complete 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) complete 017 (IBM WTX review) Minor agreed updates (Alan) complete 018 (IBM WTX review) Review generateNewLine (Alan) complete 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) complete 021 (IBM WTX review) 'EndOfData' changes (Alan) complete 022 (IBM WTX review) Unresolvable choices - infoset changes complete 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? complete 024 (IBM WTX review) StopValue clarification (Alan) complete 025 Augmented infoset and unparsing (Alan) added but needs work 026 Hidden changes complete 027 Add decimal supp to main spec complete 028 Remove extended floats complete 029 AP: enum + expression wording complete 030 All: Document new EndOfData changes complete 031 AP: Document defineTextNumber etc complete 032 AP: Document textNumberRepresenation and binaryNumberRepresentation complete complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Padding to Minimum length proposal and Implicit lengths
by Alan Powell 10 Dec '08

10 Dec '08

Mike, Steve The proposal to allow padding to minimum length on output consists of new property ouputMinLength Separate pad properties for string, number and calendar types new textxxxxPad property to control padding . Moved from from textxxxxxxJustification 1 Padding Property Name Description outputMinLength String. Only used when textStringPad, textNumberPad or textCalendarPad is ?minlength?. Specifies the minimum output length of this element using either a fixed number or an expression to refer to an element earlier in the data,. Annotation: dfdl:element (all simple types) 1.1 Properties Specific to Strings with Text representation Property Name Description textStringPad Enum Valid values ?none?, ?fixed?, ?minLength? Controls how an item is justified on output when the actual length of a text string is different from the specified length. Controls how an item is trimmed on input if textTrimKind is padChar If ?none? the string is expected to match the length. Otherwise: - If lengthKind results in a fixed length: If shorter than the specified length it is padded with the pad character. If longer than the specified length it is truncated. - If lengthKind is ?prefixed?: If the string is longer than any specified maximum length it is truncated. textStringJustification Enum Valid values ?left?, ?right?, ?center? ?center? adds equal padChars before and after the string contents if the string is too short. It adds one extra padChar before if needed. ?center? truncates equal numbers of characters before and after if the string is too long. It removes one extra character at the front of the string if needed. [OMG/CAM property justification] Annotation: dfdl:element (simple type ?string?) textStringPadCharacter String. The padding character used as the default for justification of text elements. In fixed width character sets this must contain a full single character representation. So if hex entities are used, enough hex bytes must be supplied to provided the full width of a character. In variable width character sets, this character must be a minimum-width character unless it is specified as an entity (see section TBD Entities) Annotation: dfdl:element (simple type ?string?) 1.2 Properties Specific to Number with Text representation Property Name Description textNumberPad Enum Valid values ?none?, ?fixed?, ?minLength? Controls how an item is justified on output when the actual length of a text string is different from the specified length. Controls how an item is trimmed on input if textTrimKind is padChar If ?none? the string is expected to match the length. Otherwise: - If lengthKind results in a fixed length: If shorter than the specified length it is padded with the pad character. If longer than the specified length it is truncated. - If lengthKind is ?prefixed?: If the string is longer than any specified maximum length it is truncated. textNumberJustification Enum Valid values ?left?, ?right?, ?center? ?center? adds equal padChars before and after the string contents if the string is too short. It adds one extra padChar before if needed. ?center? truncates equal numbers of characters before and after if the string is too long. It removes one extra character at the front of the string if needed. [OMG/CAM property justification] Annotation: dfdl:element (simple type ?string?) textNumberPadCharacter String. The padding character used as the default for justification of text elements. In fixed width character sets this must contain a full single character representation. So if hex entities are used, enough hex bytes must be supplied to provided the full width of a character. In variable width character sets, this character must be a minimum-width character unless it is specified as an entity (see section TBD Entities) Annotation: dfdl:element (simple type ?string?) 1.3 Properties Specific to Calendar with Text representation Property Name Description textCalendarPad Enum Valid values ?none?, ?fixed?, ?minLength? Controls how an item is justified on output when the actual length of a text string is different from the specified length. Controls how an item is trimmed on input if textTrimKind is padChar If ?none? the string is expected to match the length. Otherwise: - If lengthKind results in a fixed length: If shorter than the specified length it is padded with the pad character. If longer than the specified length it is truncated. - If lengthKind is ?prefixed?: If the string is longer than any specified maximum length it is truncated. textCalendarJustification Enum Valid values ?left?, ?right?, ?center? ?center? adds equal padChars before and after the string contents if the string is too short. It adds one extra padChar before if needed. ?center? truncates equal numbers of characters before and after if the string is too long. It removes one extra character at the front of the string if needed. [OMG/CAM property justification] Annotation: dfdl:element (simple type ?string?) textCalendarPadCharacter String. The padding character used as the default for justification of text elements. In fixed width character sets this must contain a full single character representation. So if hex entities are used, enough hex bytes must be supplied to provided the full width of a character. In variable width character sets, this character must be a minimum-width character unless it is specified as an entity (see section TBD Entities) Annotation: dfdl:element (simple type ?string?) I also completed the table of meaning of implicit lengths for different types 1.1 Lengths of simple types for lengthKind=implicit Implemented in WMB GUI helper function Simple type text (+ xml) binary String Length/maxlength n/a Float Text: numberPattern (length of positive subpattern) Zoned: TotalDigits 32 Double Text: numberPattern Zoned: TotalDigits 64 Decimal/Integer Text: numberPattern Zoned: TotalDigits Pkd/bcd: TotalDigits+1/2 Long, UnsignedLong Text: numberPattern Zoned: 19 binary: 64 Pkd/bcd: TotalDigits+1/2 Int, Unsignedint Text: numberPattern Zoned: 10 binary: 32 Pkd/bcd: TotalDigits+1/2 Short, Unsignedshort Text: numberPattern Zoned: 5 binary: 16 Pkd/bcd: TotalDigits+1/2 Byte, Unsignedbyte Text: numberPattern Zoned: 3 binary: 8 Pkd/bcd: TotalDigits+1/2 DateTime Calendarpattern Date Calendarpattern Time Calendarpattern Duration Calendarpattern Boolean Length of textBooleanTrue 32 HexBinary n/a Length/maxlength Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

3 2

DFDL: Minutes from OGF WG call, 03 December 2008
by Alan Powell 05 Dec '08

05 Dec '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 03 December 2008 Attendees Steve Hanson (IBM) Alan Powell (IBM) Appolgies Mike Beckerle(Oco) 1. Padding to minlength Discussion on best way to pad to minlength for non-string text representations Decided to split textpadCharacter into textStringPadChar, textNumberPadChar and textCalenedarPadChar so that different defaults can be set up for each type, Decided that all the ICU pattern special characters should be allowed, in particular the padding character to allow padding after the sign or currency symbol. Normal dfdl padding will be applied after the pattern has been used to generate output item. Continue on next call 3. AOB Next call 10 December Meeting closed, 13:00 GMT Actions raised at this meeting No Action Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 12/11: Updates have been sent to wtx to review. Will chase up for comments 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 26/11: discussed and AP to suggest dfdl:minlength 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 023 MB: Review Schema 1.1 024 String XML type 025 Escape schemes 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 complete 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules complete 014 (from F2F) Occurs, OccurSeparator changes complete 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) complete 017 (IBM WTX review) Minor agreed updates (Alan) complete 018 (IBM WTX review) Review generateNewLine (Alan) complete 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) complete 021 (IBM WTX review) 'EndOfData' changes (Alan) complete 022 (IBM WTX review) Unresolvable choices - infoset changes complete 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? complete 024 (IBM WTX review) StopValue clarification (Alan) complete 025 Augmented infoset and unparsing (Alan) added but needs work 026 Hidden changes complete 027 Add decimal supp to main spec complete 028 Remove extended floats complete 029 AP: enum + expression wording complete 030 All: Document new EndOfData changes complete 031 AP: Document defineTextNumber etc complete 032 AP: Document textNumberRepresenation and binaryNumberRepresentation complete complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 26 November 2008
by Alan Powell 26 Nov '08

26 Nov '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 26 November 2008 Attendees Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM) 1. Moving to XSD 1.1 as base for DFDL Continued discussion on benefit of moving to XSD 1.1 Discussed XSD assert and XPATH Restrictions - Only on complex types - Path steps are restricted to only match information items within the element being ·assessed·. Decided XSD 1.1 assert cannot replace dfdl:assert as it occurs in more places and has more function Decided not to rename dfdl:assert even though it is similar to xsd:assert as it properly describes function XSD 1.1 assert is for validation only and has no effect on dfdl parsing and unparsing. MB to investigate whether dfdl variables can be used to allow dfdl xpath paths to be restricted in the same way as XDS 1.1 as part of the existing variable action item. 2. Action Items/ Work Items. Actions 003/018 padding to minlength - discussed WTX response and AP to investigate adding dfdl:minlength Work items 013 Scoping rules. Reviewed current description in spec and agreed section on overriding should be deleted. 3. AOB Next call 3 December Meeting closed, 13:15 GMT Actions raised at this meeting No Action Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 12/11: Updates have been sent to wtx to review. Will chase up for comments 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 26/11: discussed and AP to suggest dfdl:minlength 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 023 MB: Review Schema 1.1 024 String XML type 025 Escape schemes 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules complete 014 (from F2F) Occurs, OccurSeparator changes complete 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) complete 017 (IBM WTX review) Minor agreed updates (Alan) complete 018 (IBM WTX review) Review generateNewLine (Alan) complete 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) complete 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes complete 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? complete 024 (IBM WTX review) StopValue clarification (Alan) complete 025 Augmented infoset and unparsing (Alan) 026 Hidden changes complete 027 Add decimal supp to main spec complete 028 Remove extended floats complete 029 AP: enum + expression wording complete 030 All: Document new EndOfData changes 031 AP: Document defineTextNumber etc complete 032 AP: Document textNumberRepresenation and binaryNumberRepresentation complete complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Agenda for DFDL WG call - 26-11-2008
by Alan Powell 26 Nov '08

26 Nov '08

Suggested agenda for todays call 1. XSD 1.1 2. Go through action items 3. Review current state of Spec 33 I have spent some time this week updating the spec with work items. Status below. I noticed the lengthkind/ length section needs major rework. Not sure scoping section is correct: Overload vs Override, examples look the same. 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules complete 014 (from F2F) Occurs, OccurSeparator changes complete 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) complete 017 (IBM WTX review) Minor agreed updates (Alan) complete 018 (IBM WTX review) Review generateNewLine (Alan) complete 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) complete 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes complete 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? complete 024 (IBM WTX review) StopValue clarification (Alan) complete 025 Augmented infoset and unparsing (Alan) 026 Hidden changes complete 027 Add decimal supp to main spec complete 028 Remove extended floats complete 029 AP: enum + expression wording complete 030 All: Document new EndOfData changes 031 AP: Document defineTextNumber etc complete 032 AP: Document textNumberRepresenation and binaryNumberRepresentation complete Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 15 October 2008
by Alan Powell 19 Nov '08

19 Nov '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 19 November 2008 Attendees Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM) 1. Moving to XSD 1.1 as base for DFDL Went through MB email of 12/11/08 [DFDL-WG] XML Schema 1.1 implications for DFDL Agreed the following warranted further investigation - open content - Wildcard Element Name Namespace Flexibility - needs clarification of use in dfdl - Xsd:all groups - as a replacement for unordered groups - XPath Usage - Type Tables and Alternative Conditional Types - xsd:override - support primitive datatypes and facets beyond those defined in [XML Schema: Datatypes] 5. Action Items/ Work Items. Didn't go through actions. Agree a new version of the spec would be issuesd at the end of November and all included work items would be highlighted in work item table 6. AOB Next call 26 November Meeting closed, 13:10 GMT Actions raised at this meeting No Action Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 12/11: Updates have been sent to wtx to review. Will chase up for comments 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 023 MB: Review Schema 1.1 024 String XML type 025 Escape schemes 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 *Mostly complete* 008 (from draft 32) Property precedence for writing (Steve) - *complete but under review* 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 *proposal needs writing up* 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) * in progress * 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules 014 (from F2F) Occurs, OccurSeparator changes 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) *complete* 017 (IBM WTX review) Minor agreed updates (Alan) *complete* 018 (IBM WTX review) Review generateNewLine (Alan) *complete* 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes *complete* 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? (Alan) 024 (IBM WTX review) StopValue clarification (Alan) 025 Augmented infoset and unparsing (Alan) 026 Hidden changes 027 Add decimal supp to main spec 028 Remove extended floats 029 AP: enum + expression wording 030 AP: Add hidden changes to spec 031 All: Document EndOfData changes 032 AP: Document defineTextNumber etc 032 AP: Document textNumberRepresenation and binaryNumberRepresentation Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 12 November 2008
by Alan Powell 14 Nov '08

14 Nov '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 12 November 2008 Attendees Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM) 1. Schema Component Designator specification changes. DFDL currently uses as part of the infoset to point to schema item associated with infoset item. Sandy Gao had alerted us to some changes in the SCD spec. MB reviewed these changes and concluded that the changes removed some limitations an so DFDL could now just reference the SCD spec without caveats. 2. Selectors Discussed Suman's comment on last call's decision on selectors. Reconfirmed the decision 3. Moving to XSD 1.1 as base for DFDL MB has done an initial review of the XSD 1.1 changes and identified some new features that might be used instead of DFDL extensions. Such as Alternative types instead of selectors and xpath instead of DFDL expression language. MB to review further. 4. Multi-level semantics. Discussed why schema had use this style and concluded it was not necessary for DFDL as we could build on the schema work. In passing noted the need to standard for examples in the dfdl spec for showing scoped properties Subsequent MB email In the DFDL spec., the problem exists of how to express the behavior of a construct in the situation where there are properties in scope surrounding the construct. I believe this kind of syntax can be used in the spec. to make clear the implications of scope: Properties in scope: lengthKind="implicit" representation="text" <sequence dfdl:separator=?,? dfdl:terminator=?;? dfdl:lengthKind="delimited"> // these are local <element name=?f1? type=?string? /> <element name=?f2? type=?string? /> </sequence> I would suggest the above syntax for the DFDL spec as a way to discuss property semantics. The only way properties can get in scope is by way of an enclosing complexType definition; but using that syntax explicitly begs the question of whether one means true lexical enclosure, can there be other enclosing complexTypes also, etc. The above style eliminates these issues. 5. Action Items/ Work Items. Went through actions. Updated below 6. AOB Next call 18 November Meeting closed, 13:10 GMT Actions raised at this meeting No Action 024 String XML type 025 Escape schemes 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 12/11: Updates have been sent to wtx to review. Will chase up for comments 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 023 MB: Review Schema 1.1 Closed actions: 004 MB: Mike will also author a new section in response to comments from Sandy Gao and Suman Kalia, explicitly connecting syntax with DFDL semantics 16/7: no progress 10/9: Mike not convinced this is necessary. Steve to talk to Sandy to see if it is needed for any IBM implementation that extends the existing XML parsing framework 15/10: SH to talk to Sandy 22/10: SH needs reminding of what this issue is about 12/11: Decided not needed. Closed 007 AP: enum + expression wording 13/8: no progress 10/9: No update 17/9: No update 22/10: No update 12/11: converted to work item 014 SH: Use cases for lengthKind pattern on binary fields. 10/9: Not done 17/9: No update 22/10: SH not sure he can provide a concrete use case 12/11: SH had identified need for pattern of complex structures with binary comtent which already included. Decided to allow patterns on simple binary fields for symetry. Closed 016 AP: Add hidden changes to spec 22/10: No update 12/11:converted to work item 017 All: Review EndOfData changes 22/10: SH response mailed to Mike 5/11: Discussed on call and design agreed 12/11: converted to work item 019 AP: Document defineTextNumber etc 22/10: No update 12/11:converted to work item 021 AP: Document textNumberRepresenation and binaryNumberRepresentation 22/10: No update 12/11:converted to work item 022 SH: Talk to Sandy Gao to assess use of schema 1.1 22/10: Mail sent to Sandy 12/11: track as action 023 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 *Mostly complete* 008 (from draft 32) Property precedence for writing (Steve) - *complete but under review* 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 *proposal needs writing up* 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) * in progress * 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules 014 (from F2F) Occurs, OccurSeparator changes 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) *complete* 017 (IBM WTX review) Minor agreed updates (Alan) *complete* 018 (IBM WTX review) Review generateNewLine (Alan) *complete* 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes *complete* 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? (Alan) 024 (IBM WTX review) StopValue clarification (Alan) 025 Augmented infoset and unparsing (Alan) 026 Hidden changes 027 Add decimal supp to main spec 028 Remove extended floats 029 AP: enum + expression wording 030 AP: Add hidden changes to spec 031 All: Document EndOfData changes 032 AP: Document defineTextNumber etc 032 AP: Document textNumberRepresenation and binaryNumberRepresentation Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0