DFDL: Minutes from OGF WG call, 05 Octobet 2008
Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 5 November 2008 Attendees Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM) 1. End Of Parent Discussed last proposal and subsequent emails. Decided: Only last element in a sequence can have lenghtkind=EndOfParent Sequence can also have lengthKind=EndOfParent - means take length from parent sequence. LengthKind=endOfParent on outermost sequence means end of data stream. So for the last field in a definition to extend to the end of the data stream the element has lengthKind="endOfParent" and the enclosing sequence, and all parent sequences including outermost also have lengthKind="endOfParent" Examples 1. Last element to end of data stream <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 2. last element to fixed length container <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="100" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 3. last element to container terminator <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="delimited" terminator="$" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> Isn't this the same as "last " with lengthKind = "delimited"? 2. Selectors Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision - If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used. Reiterated that selectors are qnames. 3. Moving to XSD 1.1 as base for DFDL Benfits: Can use xsd:all instead of introducing unordered sequences. weakened wildcard rules. Agreed in principle. MB to investigate to look at the other changes in Schema 1.1 to check effect on other dfdl functions and restrictions 4. Not dicussed SCD spec changes Multi-level semntics 2. AOB Next call 12 November Meeting closed, 13:10 GMT Actions raised at this meeting No Action 023 MB: Review Schema 1.1 Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 004 MB: Mike will also author a new section in response to comments from Sandy Gao and Suman Kalia, explicitly connecting syntax with DFDL semantics 16/7: no progress 10/9: Mike not convinced this is necessary. Steve to talk to Sandy to see if it is needed for any IBM implementation that extends the existing XML parsing framework 15/10: SH to talk to Sandy 22/10: SH needs reminding of what this issue is about 007 AP: enum + expression wording 13/8: no progress 10/9: No update 17/9: No update 22/10: No update 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 014 SH: Use cases for lengthKind pattern on binary fields. 10/9: Not done 17/9: No update 22/10: SH not sure he can provide a concrete use case 016 AP: Add hidden changes to spec 22/10: No update 017 All: Review EndOfData changes 22/10: SH response mailed to Mike 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 019 AP: Document defineTextNumber etc 22/10: No update 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 021 AP: Document textNumberRepresenation and binaryNumberRepresentation 22/10: No update 022 SH: Talk to Sandy Gao to assess use of schema 1.1 22/10: Mail sent to Sandy Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 *Mostly complete* 008 (from draft 32) Property precedence for writing (Steve) - *complete but under review* 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 *proposal needs writing up* 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) * in progress * 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules 014 (from F2F) Occurs, OccurSeparator changes 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) *complete* 017 (IBM WTX review) Minor agreed updates (Alan) *complete* 018 (IBM WTX review) Review generateNewLine (Alan) *complete* 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes *complete* 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? (Alan) 024 (IBM WTX review) StopValue clarification (Alan) 025 Augmented infoset and unparsing (Alan) 026 Hidden changes 027 Add decimal supp to main spec 028 Remove extended floats Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Hi Alan You asked for your end of parent example 3: Isn't this the same as "last " with lengthKind = "delimited"? Yes it is. What end of parent provides is the ability to switch off delimiter scanning, but only if there are no in-scope delimiters. Here there is a delimiter in-scope, so we can't optimise away scanning, so the net effect is the same as delimited. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh@uk.ibm.com Phone (+44)/(0) 1962-815848 Alan Powell/UK/IBM@IBMGB Sent by: dfdl-wg-bounces@ogf.org 06/11/2008 16:54 To dfdl-wg@ogf.org cc Subject [DFDL-WG] DFDL: Minutes from OGF WG call, 05 Octobet 2008 Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 5 November 2008 Attendees Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM) 1. End Of Parent Discussed last proposal and subsequent emails. Decided: Only last element in a sequence can have lenghtkind=EndOfParent Sequence can also have lengthKind=EndOfParent - means take length from parent sequence. LengthKind=endOfParent on outermost sequence means end of data stream. So for the last field in a definition to extend to the end of the data stream the element has lengthKind="endOfParent" and the enclosing sequence, and all parent sequences including outermost also have lengthKind="endOfParent" Examples 1. Last element to end of data stream <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 2. last element to fixed length container <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="100" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 3. last element to container terminator <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="delimited" terminator="$" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> Isn't this the same as "last " with lengthKind = "delimited"? 2. Selectors Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision - If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used. Reiterated that selectors are qnames. 3. Moving to XSD 1.1 as base for DFDL Benfits: Can use xsd:all instead of introducing unordered sequences. weakened wildcard rules. Agreed in principle. MB to investigate to look at the other changes in Schema 1.1 to check effect on other dfdl functions and restrictions 4. Not dicussed SCD spec changes Multi-level semntics 2. AOB Next call 12 November Meeting closed, 13:10 GMT Actions raised at this meeting No Action 023 MB: Review Schema 1.1 Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 004 MB: Mike will also author a new section in response to comments from Sandy Gao and Suman Kalia, explicitly connecting syntax with DFDL semantics 16/7: no progress 10/9: Mike not convinced this is necessary. Steve to talk to Sandy to see if it is needed for any IBM implementation that extends the existing XML parsing framework 15/10: SH to talk to Sandy 22/10: SH needs reminding of what this issue is about 007 AP: enum + expression wording 13/8: no progress 10/9: No update 17/9: No update 22/10: No update 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 014 SH: Use cases for lengthKind pattern on binary fields. 10/9: Not done 17/9: No update 22/10: SH not sure he can provide a concrete use case 016 AP: Add hidden changes to spec 22/10: No update 017 All: Review EndOfData changes 22/10: SH response mailed to Mike 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 019 AP: Document defineTextNumber etc 22/10: No update 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 021 AP: Document textNumberRepresenation and binaryNumberRepresentation 22/10: No update 022 SH: Talk to Sandy Gao to assess use of schema 1.1 22/10: Mail sent to Sandy Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 *Mostly complete* 008 (from draft 32) Property precedence for writing (Steve) - *complete but under review* 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 *proposal needs writing up* 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) * in progress * 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules 014 (from F2F) Occurs, OccurSeparator changes 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) *complete* 017 (IBM WTX review) Minor agreed updates (Alan) *complete* 018 (IBM WTX review) Review generateNewLine (Alan) *complete* 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes *complete* 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? (Alan) 024 (IBM WTX review) StopValue clarification (Alan) 025 Augmented infoset and unparsing (Alan) 026 Hidden changes 027 Add decimal supp to main spec 028 Remove extended floats Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
2. Selectors
Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used
Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision
- If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used.
Reiterated that selectors are qnames.
Alan - I think it will add more complexity and it will be difficult for the user to comprehend the exact set of annotations that would be applicable on the element. This becomes more complicated when you take inheritance of annotations into account. Say grandparent A has annotation with selector xyz , parent B & its child C has annotation with no selector. When processing for selector xyz, which annotations will be applicable for child C - will it inherit annotations from grandparent A (which has same selector) or parent B (which has annotation with no selector specified) ?. What if grand child C had no dfdl annotation specified ? Suman Kalia IBM Toronto Lab WMB Toolkit Architect and Development Lead WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia@ca.ibm.com From: Alan Powell <alan_powell@uk.ibm.com> To: dfdl-wg@ogf.org Date: 11/06/2008 11:55 AM Subject: [DFDL-WG] DFDL: Minutes from OGF WG call, 05 Octobet 2008 Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 5 November 2008 Attendees Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM) 1. End Of Parent Discussed last proposal and subsequent emails. Decided: Only last element in a sequence can have lenghtkind=EndOfParent Sequence can also have lengthKind=EndOfParent - means take length from parent sequence. LengthKind=endOfParent on outermost sequence means end of data stream. So for the last field in a definition to extend to the end of the data stream the element has lengthKind="endOfParent" and the enclosing sequence, and all parent sequences including outermost also have lengthKind="endOfParent" Examples 1. Last element to end of data stream <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 2. last element to fixed length container <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="100" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 3. last element to container terminator <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="delimited" terminator="$" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> Isn't this the same as "last " with lengthKind = "delimited"? 2. Selectors Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision - If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used. Reiterated that selectors are qnames. 3. Moving to XSD 1.1 as base for DFDL Benfits: Can use xsd:all instead of introducing unordered sequences. weakened wildcard rules. Agreed in principle. MB to investigate to look at the other changes in Schema 1.1 to check effect on other dfdl functions and restrictions 4. Not dicussed SCD spec changes Multi-level semntics 2. AOB Next call 12 November Meeting closed, 13:10 GMT Actions raised at this meeting No Action 023 MB: Review Schema 1.1 Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 004 MB: Mike will also author a new section in response to comments from Sandy Gao and Suman Kalia, explicitly connecting syntax with DFDL semantics 16/7: no progress 10/9: Mike not convinced this is necessary. Steve to talk to Sandy to see if it is needed for any IBM implementation that extends the existing XML parsing framework 15/10: SH to talk to Sandy 22/10: SH needs reminding of what this issue is about 007 AP: enum + expression wording 13/8: no progress 10/9: No update 17/9: No update 22/10: No update 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 014 SH: Use cases for lengthKind pattern on binary fields. 10/9: Not done 17/9: No update 22/10: SH not sure he can provide a concrete use case 016 AP: Add hidden changes to spec 22/10: No update 017 All: Review EndOfData changes 22/10: SH response mailed to Mike 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 019 AP: Document defineTextNumber etc 22/10: No update 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 021 AP: Document textNumberRepresenation and binaryNumberRepresentation 22/10: No update 022 SH: Talk to Sandy Gao to assess use of schema 1.1 22/10: Mail sent to Sandy Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 *Mostly complete* 008 (from draft 32) Property precedence for writing (Steve) - *complete but under review* 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 *proposal needs writing up* 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) * in progress * 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules 014 (from F2F) Occurs, OccurSeparator changes 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) *complete* 017 (IBM WTX review) Minor agreed updates (Alan) *complete* 018 (IBM WTX review) Review generateNewLine (Alan) *complete* 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes *complete* 023 (IBM WTX review) separatorKind=’prefix’ ‘infix’ or ‘postfix’ (Alan) 024 (IBM WTX review) StopValue clarification (Alan) 025 Augmented infoset and unparsing (Alan) 026 Hidden changes 027 Add decimal supp to main spec 028 Remove extended floats Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg
Suman We discussed the options on the call and decided that forcing multiple formats statement on every element (which is what WMB does) was too cumbersome. Selectors do not change inheritance rules at all so your questions are the same whether selectors are used or not. We felt the selectors rule were fairly simple and once it has been decided which format block to use normal scoping rules apply. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: Suman Kalia <kalia@ca.ibm.com> To: Alan Powell/UK/IBM@IBMGB Cc: dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org Date: 06/11/2008 23:21 Subject: Re: [DFDL-WG] DFDL: Minutes from OGF WG call, 05 Octobet 2008
2. Selectors
Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used
Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision
- If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used.
Reiterated that selectors are qnames.
Alan - I think it will add more complexity and it will be difficult for the user to comprehend the exact set of annotations that would be applicable on the element. This becomes more complicated when you take inheritance of annotations into account. Say grandparent A has annotation with selector xyz , parent B & its child C has annotation with no selector. When processing for selector xyz, which annotations will be applicable for child C - will it inherit annotations from grandparent A (which has same selector) or parent B (which has annotation with no selector specified) ?. What if grand child C had no dfdl annotation specified ? Suman Kalia IBM Toronto Lab WMB Toolkit Architect and Development Lead WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia@ca.ibm.com From: Alan Powell <alan_powell@uk.ibm.com> To: dfdl-wg@ogf.org Date: 11/06/2008 11:55 AM Subject: [DFDL-WG] DFDL: Minutes from OGF WG call, 05 Octobet 2008 Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 5 November 2008 Attendees Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM) 1. End Of Parent Discussed last proposal and subsequent emails. Decided: Only last element in a sequence can have lenghtkind=EndOfParent Sequence can also have lengthKind=EndOfParent - means take length from parent sequence. LengthKind=endOfParent on outermost sequence means end of data stream. So for the last field in a definition to extend to the end of the data stream the element has lengthKind="endOfParent" and the enclosing sequence, and all parent sequences including outermost also have lengthKind="endOfParent" Examples 1. Last element to end of data stream <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 2. last element to fixed length container <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="100" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 3. last element to container terminator <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="delimited" terminator="$" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> Isn't this the same as "last " with lengthKind = "delimited"? 2. Selectors Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision - If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used. Reiterated that selectors are qnames. 3. Moving to XSD 1.1 as base for DFDL Benfits: Can use xsd:all instead of introducing unordered sequences. weakened wildcard rules. Agreed in principle. MB to investigate to look at the other changes in Schema 1.1 to check effect on other dfdl functions and restrictions 4. Not dicussed SCD spec changes Multi-level semntics 2. AOB Next call 12 November Meeting closed, 13:10 GMT Actions raised at this meeting No Action 023 MB: Review Schema 1.1 Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 004 MB: Mike will also author a new section in response to comments from Sandy Gao and Suman Kalia, explicitly connecting syntax with DFDL semantics 16/7: no progress 10/9: Mike not convinced this is necessary. Steve to talk to Sandy to see if it is needed for any IBM implementation that extends the existing XML parsing framework 15/10: SH to talk to Sandy 22/10: SH needs reminding of what this issue is about 007 AP: enum + expression wording 13/8: no progress 10/9: No update 17/9: No update 22/10: No update 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 014 SH: Use cases for lengthKind pattern on binary fields. 10/9: Not done 17/9: No update 22/10: SH not sure he can provide a concrete use case 016 AP: Add hidden changes to spec 22/10: No update 017 All: Review EndOfData changes 22/10: SH response mailed to Mike 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 019 AP: Document defineTextNumber etc 22/10: No update 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 021 AP: Document textNumberRepresenation and binaryNumberRepresentation 22/10: No update 022 SH: Talk to Sandy Gao to assess use of schema 1.1 22/10: Mail sent to Sandy Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 *Mostly complete* 008 (from draft 32) Property precedence for writing (Steve) - *complete but under review* 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 *proposal needs writing up* 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) * in progress * 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules 014 (from F2F) Occurs, OccurSeparator changes 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) *complete* 017 (IBM WTX review) Minor agreed updates (Alan) *complete* 018 (IBM WTX review) Review generateNewLine (Alan) *complete* 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes *complete* 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? (Alan) 024 (IBM WTX review) StopValue clarification (Alan) 025 Augmented infoset and unparsing (Alan) 026 Hidden changes 027 Add decimal supp to main spec 028 Remove extended floats Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
The rules seem pretty simple to me. Use selector rules first to erase all unused annotations. The scoping is applied as before. Perhaps this algorithm wasn't stated? It is definitely true if you entangle the two so that scoping involves selector decisions then it seems more complex. We should keep the two cleanly separated. Selectors should be thought of as a preprocessor. ...mikeb On Nov 6, 2008, at 6:22 PM, Suman Kalia <kalia@ca.ibm.com> wrote:
2. Selectors
Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used
Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision
- If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used.
Reiterated that selectors are qnames.
Alan - I think it will add more complexity and it will be difficult for the user to comprehend the exact set of annotations that would be applicable on the element. This becomes more complicated when you take inheritance of annotations into account. Say grandparent A has annotation with selector xyz , parent B & its child C has annotation with no selector. When processing for selector xyz, which annotations will be applicable for child C - will it inherit annotations from grandparent A (which has same selector) or parent B (which has annotation with no selector specified) ?. What if grand child C had no dfdl annotation specified ?
Suman Kalia IBM Toronto Lab WMB Toolkit Architect and Development Lead WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia@ca.ibm.com
From: Alan Powell <alan_powell@uk.ibm.com> To: dfdl-wg@ogf.org Date: 11/06/2008 11:55 AM Subject: [DFDL-WG] DFDL: Minutes from OGF WG call, 05 Octobet 2008
Open Grid Forum: Data Format Description Language Working Group
Weekly Working Group Conference Call 13:00 GMT, 5 November 2008
Attendees
Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM)
1. End Of Parent Discussed last proposal and subsequent emails.
Decided: Only last element in a sequence can have lenghtkind=EndOfParent Sequence can also have lengthKind=EndOfParent - means take length from parent sequence. LengthKind=endOfParent on outermost sequence means end of data stream. So for the last field in a definition to extend to the end of the data stream the element has lengthKind="endOfParent" and the enclosing sequence, and all parent sequences including outermost also have lengthKind="endOfParent"
Examples
1. Last element to end of data stream <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent" /> </xsd:appinfo> </xsd:annotation>
<xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence>
2. last element to fixed length container
<xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="100" /> </xsd:appinfo> </xsd:annotation>
<xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence>
3. last element to container terminator
<xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="delimited" terminator="$" /> </xsd:appinfo> </xsd:annotation>
<xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence>
Isn't this the same as "last " with lengthKind = "delimited"?
2. Selectors
Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used
Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision
- If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used.
Reiterated that selectors are qnames.
3. Moving to XSD 1.1 as base for DFDL
Benfits: Can use xsd:all instead of introducing unordered sequences. weakened wildcard rules.
Agreed in principle. MB to investigate to look at the other changes in Schema 1.1 to check effect on other dfdl functions and restrictions
4. Not dicussed
SCD spec changes Multi-level semntics
2. AOB Next call 12 November
Meeting closed, 13:10 GMT Actions raised at this meeting
No Action 023 MB: Review Schema 1.1 Current Actions:
No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 004 MB: Mike will also author a new section in response to comments from Sandy Gao and Suman Kalia, explicitly connecting syntax with DFDL semantics 16/7: no progress 10/9: Mike not convinced this is necessary. Steve to talk to Sandy to see if it is needed for any IBM implementation that extends the existing XML parsing framework 15/10: SH to talk to Sandy 22/10: SH needs reminding of what this issue is about 007 AP: enum + expression wording 13/8: no progress 10/9: No update 17/9: No update 22/10: No update 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 014 SH: Use cases for lengthKind pattern on binary fields. 10/9: Not done 17/9: No update 22/10: SH not sure he can provide a concrete use case 016 AP: Add hidden changes to spec 22/10: No update 017 All: Review EndOfData changes 22/10: SH response mailed to Mike 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 019 AP: Document defineTextNumber etc 22/10: No update 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 021 AP: Document textNumberRepresenation and binaryNumberRepresentation 22/10: No update 022 SH: Talk to Sandy Gao to assess use of schema 1.1 22/10: Mail sent to Sandy Closed actions:
033 Work items:
No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 *Mostly complete* 008 (from draft 32) Property precedence for writing (Steve) - *complete but under review* 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 *proposal needs writing up* 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) * in progress * 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules 014 (from F2F) Occurs, OccurSeparator changes 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) *complete* 017 (IBM WTX review) Minor agreed updates (Alan) *complete* 018 (IBM WTX review) Review generateNewLine (Alan) *complete* 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes *complete* 023 (IBM WTX review) separatorKind=’prefix’ ‘infix’ or ‘postfix’ (Alan) 024 (IBM WTX review) StopValue clarification (Alan) 025 Augmented infoset and unparsing (Alan) 026 Hidden changes 027 Add decimal supp to main spec 028 Remove extended floats
Alan Powell
MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailmYW4vbGlzdGluZm8vZGZkbC13ZzwvZm9udD48L3R0PjwvYT4NCjxw... t size=3>
-- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg
-- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg
Hi Suman You and me discussed your simpler rule for applying selectors, on the DFDL OGF WG call of 22nd October. I was ok at the time with it, but while writing up the minutes I became concerned that it forces you to add a new DFDL annotation to all objects in the schema, and to any objects in included/imported schemas, when a new selector is added. Hence my example in the minutes. Alan, Mike and me discussed selectors on the 5th Nov call and agreed that having to apply annotations on all objects was too cumbersome, especially for the include/import scenario. Hence we prefer the selector proposal as originally outlined. I don't think inheritance of annotations is an issue. Firstly the selector property itself is never inherited. Secondly, selectors are processed before inheritance - any annotations with explicit selectors that do not match the provided selector are stripped out of the schema - then the scoping rules are applied. And thirdly, scoping is simpler anyway these days, as it is only complex & simple types that can scope DFDL properties. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh@uk.ibm.com Phone (+44)/(0) 1962-815848 Suman Kalia <kalia@ca.ibm.com> Sent by: dfdl-wg-bounces@ogf.org 06/11/2008 23:22 To Alan Powell/UK/IBM@IBMGB cc dfdl-wg@ogf.org, dfdl-wg-bounces@ogf.org Subject Re: [DFDL-WG] DFDL: Minutes from OGF WG call, 05 Octobet 2008
2. Selectors
Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used
Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision
- If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used.
Reiterated that selectors are qnames.
Alan - I think it will add more complexity and it will be difficult for the user to comprehend the exact set of annotations that would be applicable on the element. This becomes more complicated when you take inheritance of annotations into account. Say grandparent A has annotation with selector xyz , parent B & its child C has annotation with no selector. When processing for selector xyz, which annotations will be applicable for child C - will it inherit annotations from grandparent A (which has same selector) or parent B (which has annotation with no selector specified) ?. What if grand child C had no dfdl annotation specified ? Suman Kalia IBM Toronto Lab WMB Toolkit Architect and Development Lead WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia@ca.ibm.com From: Alan Powell <alan_powell@uk.ibm.com> To: dfdl-wg@ogf.org Date: 11/06/2008 11:55 AM Subject: [DFDL-WG] DFDL: Minutes from OGF WG call, 05 Octobet 2008 Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 13:00 GMT, 5 November 2008 Attendees Steve Hanson (IBM) Mike Beckerle(Oco) Alan Powell (IBM) 1. End Of Parent Discussed last proposal and subsequent emails. Decided: Only last element in a sequence can have lenghtkind=EndOfParent Sequence can also have lengthKind=EndOfParent - means take length from parent sequence. LengthKind=endOfParent on outermost sequence means end of data stream. So for the last field in a definition to extend to the end of the data stream the element has lengthKind="endOfParent" and the enclosing sequence, and all parent sequences including outermost also have lengthKind="endOfParent" Examples 1. Last element to end of data stream <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 2. last element to fixed length container <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="100" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> 3. last element to container terminator <xsd:sequence> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="delimited" terminator="$" /> </xsd:appinfo> </xsd:annotation> <xs:element name="mandt" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="explicit" length="3"/> </xsd:appinfo> </xsd:annotation> </xsd:element> .... <xs:element name="last" type="xsd:string"/> <xsd:annotation> <xsd:appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format lengthKind="endOfParent"/> </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:sequence> Isn't this the same as "last " with lengthKind = "delimited"? 2. Selectors Discussed Suman's suggestion from last weeks call. - If a selector is supplied, then only dfdl:format blocks matching that selector are used - If a selector is not supplied, then only dfdl:format blocks without a selector are used Decided this is too restrictive as it requires a format block for each selector on every item even if they define the same properties so stayed with previous decision - If selector is supplied and matches one of the dfdl:format blocks then that block is used. - Otherwise the dfdl:format block with no selector is used. Reiterated that selectors are qnames. 3. Moving to XSD 1.1 as base for DFDL Benfits: Can use xsd:all instead of introducing unordered sequences. weakened wildcard rules. Agreed in principle. MB to investigate to look at the other changes in Schema 1.1 to check effect on other dfdl functions and restrictions 4. Not dicussed SCD spec changes Multi-level semntics 2. AOB Next call 12 November Meeting closed, 13:10 GMT Actions raised at this meeting No Action 023 MB: Review Schema 1.1 Current Actions: No Action 003 AP: Update spec from WTX document 13/8: All resolved issues have been added to spec. Remaining unresolved items need discussion 10/9: Alan will pull out the remaining unresolved items and circulate for discussion next week 17/9: Doc distributed and partially discussed 15/10: All updateds complete except minlength 22/10: No update 004 MB: Mike will also author a new section in response to comments from Sandy Gao and Suman Kalia, explicitly connecting syntax with DFDL semantics 16/7: no progress 10/9: Mike not convinced this is necessary. Steve to talk to Sandy to see if it is needed for any IBM implementation that extends the existing XML parsing framework 15/10: SH to talk to Sandy 22/10: SH needs reminding of what this issue is about 007 AP: enum + expression wording 13/8: no progress 10/9: No update 17/9: No update 22/10: No update 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 014 SH: Use cases for lengthKind pattern on binary fields. 10/9: Not done 17/9: No update 22/10: SH not sure he can provide a concrete use case 016 AP: Add hidden changes to spec 22/10: No update 017 All: Review EndOfData changes 22/10: SH response mailed to Mike 018 AP: Agree padding to minimum length 15/10: Needs documenting 22/10: No update 019 AP: Document defineTextNumber etc 22/10: No update 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 021 AP: Document textNumberRepresenation and binaryNumberRepresentation 22/10: No update 022 SH: Talk to Sandy Gao to assess use of schema 1.1 22/10: Mail sent to Sandy Closed actions: 033 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 004 Selectors (Suman/Alan) - Apr 30, 2008 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 *Mostly complete* 008 (from draft 32) Property precedence for writing (Steve) - *complete but under review* 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 *proposal needs writing up* 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) * in progress * 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 013 (from F2F) New scoping rules 014 (from F2F) Occurs, OccurSeparator changes 015 (from F2F) choices and Output (Mike) 016 (from F2F) xpath forward references (Alan) *complete* 017 (IBM WTX review) Minor agreed updates (Alan) *complete* 018 (IBM WTX review) Review generateNewLine (Alan) *complete* 020 (IBM WTX review) Special value for zero seen eg 'zero'. (Steve) 021 (IBM WTX review) 'EndOfData' changes (Alan) 022 (IBM WTX review) Unresolvable choices - infoset changes *complete* 023 (IBM WTX review) separatorKind=?prefix? ?infix? or ?postfix? (Alan) 024 (IBM WTX review) StopValue clarification (Alan) 025 Augmented infoset and unparsing (Alan) 026 Hidden changes 027 Add decimal supp to main spec 028 Remove extended floats Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell@uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg -- dfdl-wg mailing list dfdl-wg@ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
I've given the XML Schema 1.1 changes a quick review and created the attached document. More work is required. There are a few features in here which we need to really understand not because they mess us up, but because they have the potential to greatly simplify our lives. In particular, they have paths in XSD 1.1, and therefore an expression language semantics, a notion of what it means for a path in one place of a schema to refer to a data element defined by another part of the schema. They have an XSD based infoset that is used to explain this. Hopefully we can leverage all this work, and therefore eliminate description of how this stuff works that is specific to DFDL. There is also conditional type alternatives for elements, and this potentially has impact on selectors - i.e., we may not need them. But it requires a careful look. ...mike <mailto:dfdl-wg@ogf.org>
participants (5)
-
Alan Powell -
DFDL -
Mike Beckerle
-
Steve Hanson -
Suman Kalia