April 2009 - dfdl-wg - lists.ogf.org

Minutes: OGF DFDL Working Group Call, April-15-2009
by Alan Powell 16 Apr '09

16 Apr '09

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, April-15-2009 Attendees Suman Kalia (IBM) Steve Hanson (IBM) Mike Beckerle (Oco) Alan Powell (IBM) Dave Glick (drac) Apologies 1. Escape Schemes Alan has mailed latest refinement (V2) just before the call. MB sent comments Agreed to drop escapeKind='escapeBlock2' as it was quite resctrictive and was primarily intended for XML style escaping which should be covered by representation='XML'. We got bogged down in individual use cases so AP agreed to document a set of use cases 2. 'Floating' definitions SH had sent out an example of need for floating definitions. The NTE segment in older X12 formats can appear multiple times anywhere within a transaction. This could be modelled in dfdl using - an array of choices which has all the possible fields as choices. This would mean that the fields other than the NTE segment could appear in any order so the document could not be validated using just the schema. - Inserting an optional NTE segment between every other segment. Such a schema would violate UPA rules if any other segments were optional. SH will investigate further to find out if this is used in versions of X12 that should be supported by DFDL v1. 3. Recursive use of DFDL for variable markup SH has sent a use case which WTX solved using 'variable markup'. The use case is from EDI. EDI transactions consist of an initial header segment which defines, among other things, the separator that is used by the data segments that follow. The problem is that EDI transactions may be processed in their entirety, or individual data segments may be processed without the header segment. For the former case, DFDL supports this fine, using an XPath expression to locate the separator, which is defined as an element, the simple type of which enumerates the allowable values, enabling validation. But for the latter case, the XPath expression won't resolve, as there is no header. An explicit dfdl:separator property could be used instead, being a space separated list of all the allowable values - but that then duplicates the separator element enums, leaving a maintenance problem. To be discussed on next call. 4. AOB Issues to be discussed - Suman has identified a identified that the short and long form properties require all dfdl properties to be defined at least twice in the schema for DFDL. - The scoping rules for annotations other than dfdl:format are not specified and need to be clarified. Actions updated below. Next call 22 April 14:00 UK Meeting closed, 15:15 Actions raised at this meeting No Action Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 04/02: changed from locale to specific properties 18/2: Need more investigation of ICU strict/lax behaviour. 08/04: Not discussed 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 11/02: proposal largely agreed. SH to make minor changes 18/02: AP to document unsigned type behaviour 25/02: no progress 08/04: Not discussed 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 04/02 Call arranged for Friday 11/02: Call took place. Identified useful changes. Consolidate with previous list. 04/03: decided to stay on Schema 1.0. 08/04: Not discussed 024 String XML type 08/04: Not discussed 025 Escape schemes 21/1: discussed requirements 04/02: AP/SH to describe behaviour for known length text fields. Need to discuss if comment escapes should be supported. 11/02 new draft distributed: 18/02: SH up document concerns 25/02: SH and AP have refined proposal ready for approval. 04/03: SH and AP have further refined proposal. 11/03: discussed. suggested a simplified proposal be evaluated. 18/03: SH and AP had further discussions on simplified proposal 08/04: See minutes, review in detail for next call 15/04: See minutes, review for next call 026 SH: Envelopes and Payloads 08/04: Not discussed explicity, but recursive use of DFDL is tied up with this 027 Property precedence tables 08/04: Not discussed 028 SH: Variable markup 08/04: Discussed briefly at end of call, IBM to see whether there any use cases that require recursive use of DFDL. 15/04: Used case was distributed and will be discussed on next call. 029 valueCalc (output length calculation) 08/04: Not discussed 032 DG: Investigate compatibility between DFDL infoset and XDM 08/04: No update 033 AP/TK: Assert/Discriminator semantics. AP to document. TK to check uses of discriminator besides choice. 08/04: In progress within IBM 034 AP: Remove redundant properties, correct old examples 08/04: No update 035 AP: Add validation ranges to spec, update specialized annotations in spec. 08/04: Raised. For draft 0.34 036 SH: Provide use case for floating component in a sequence 08/04: Raised 15/04: Use case sent and discussed. SH to do further investigation Closed actions: Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work 026 Remove duration ã Copyright IBM Corp. 1998, 2007 All Rights Reserved Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL Schema - Mulitple forms of Annotation syntax
by Suman Kalia 15 Apr '09

15 Apr '09

Following forms of annotations syntax is supported on DFDL schema as per the 1.33 spec 1. Shot form syntax <xs:element name="foo" type="xs:int" maxOccurs="unbounded" dfdl:representation="text" dfdl:initiator="{" dfdl:terminator=”}”/> Note: It requires all properties to be defined as global attributes in schema as each attribute has to be uniquely identifiable and accessible within the DFDL namespace 2. Long form syntax ( used for dfdl:format and specialized annotations such as dfdl:element etc) <xs:element name="foo" type="xs:int" maxOccurs="unbounded"> <xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”> <dfdl:format representation="text" initiator=”{“ terminator="}"/> </xs:appinfo></xs:annotation> </xs:element> Note: Attributes specified here are not qualified with dfdl namespace prefix. It means the attributes have to be defined locally within the complex type are not accessible directly (like in the short form syntax) 3. Long form property representation to ease syntactic expression difficulties with data in attribute values: <xs:annotation> <xs:appinfo source=”http://www.ogf.org/dfdl/”> <dfdl:format> <dfdl:property name='encoding'>utf-8</dfdl:property> <dfdl:property name='separator'>\n</dfdl:property> <dfdl:property name=’initiator’><[CDATA[<!-- ]]></dfdl:property> </dfdl:format> </xs:appinfo> </xs:annotation> Note: The name attribute will be enumeration of base name of all DFDL properties (ie without namespace prefix). On the specialized annotations (ie dfdl:element) we will also need to have a repeating child element definition for dfdl:property. This annotation will appear on specialized annotation element for dfdl attributes whose value cannot be stored as XML attribute. PS: Alan - when you update the spec for specialized annotation please mention this use case and put an example also. To support syntax form 1 & 2, it requires defining all attribute definition for each DFDL property multiple times ie one global attribute and a local attribute in the complex type of each long form annotation element. Using Attribute groups we can address the latter issue ie define related local attributes once in the attribute group and include the attribute group in one or more complex type. But it still requires defining the attribute definition for each DFDL property atleast twice (ie one global and one local attribute definition). This duplication can potentially lead to inconsistency and inadvertently updating one definition and not the other. One option that we can consider to remove this redundancy is to define only the global attribute definition for each DFDL property and include it through attribute reference in each of the complex type of dfdl:format and specialized annotations. The downside of this approach is that attributes will appear qualified in the dfdl:format and specialized annotation elements. Given below is how the dfdl:format annotation will appear in the user defined schema where each attribute is prefixed with dfdl (hilited in green). <xs:element name="foo" type="xs:int" maxOccurs="unbounded"> <xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”> <dfdl:format dfdl:representation="text" dfdl:initiator=”{“ dfdl: terminator="}"/> </xs:appinfo></xs:annotation> </xs:element> From technical perspective (e.g. schema integrity , avoiding duplication definitions etc.), this is better but it does have usability issue. Suman Kalia IBM Toronto Lab WMB Toolkit Architect and Development Lead WebSphere Business Integration Application Connectivity Tools http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h… Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com

1 0

Simplified EscapeScheme proposal
by Alan Powell 15 Apr '09

15 Apr '09

While working through the previous generalized proposal for escape schemes it became obvious that it was becoming overly complicated for what are believed to be a small number of escaping mechanisms. It had become particularly difficult to define the rules for when escaping should be used 'when needed' The attached proposal is much simplified and covers just three variants of escaping mechanisms. - The use of a single escape character to cause the next character to be interpreted literally. The escape character itself is escaped by the escape escape character. - The use of a pair of escape strings to cause the enclosed group of characters to be interpreted literally. The ending escape string is escaped by an escape escape character. - The use of a pair of escape strings to cause the enclosed group of characters to be interpreted literally. An alternate pair of escape strings is used when the data contains the first ending escape string. I have also introduced an extraEscapedCharacters property which is a list of additional single characters which must be escaped as a get out for other mechanisms Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

DFDL WG Call 2009-04-15: Agenda
by Alan Powell 15 Apr '09

15 Apr '09

Agenda: Escape Schemes - review details. latest document attached 'Floating' definitions - discuss IBM use case Finalizing DFDL v1 Actions Alan Powell Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Floating Components in EDI X12
by Steve Hanson 15 Apr '09

15 Apr '09

>From last week's call: 6. 'Floating' definitions A known element, the position of which can be anywhere in a sequence of other elements - is this something DFDL needs to support? Capability is offered by IBM's WTX product. Can be used for comments, but DFDL plans to handle comments post 1.0 using an explicit mechanism or using layering. Real purpose of floating component is for older EDI formats where there is a segment that can appear anywhere, and can appear any number of times. Action raised for IBM to provide a concrete example for discussion. The issue for DFDL is how does a floating component appear in the DFDL infoset, and how does it validate in the sequence. One possibility is a property dfdl:floating=yes/no and if an element has that property set, it can be expected anywhere when parsing, but appears in the correct point in the sequence in the parsed infoset, On unparsing it must appear in in the correct point in the sequence, and is output in that place. 036 SH: Provide use case for floating component in a sequence 08/04: Raised Example below, kindly supplied by Stephanie Fetzer of IBM. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 FLOATING COMPONENTS At one point In X12 segments could be: M (Mandatory) This segment must be present O (Optional) Trading Partners decide on the use of this segment F (Floating) This segment may appear anywhere within the transaction. This designator has been discontinued. The NTE segment was the only usage of a floating component (meaning that it could appear anywhere within the bounds of the transaction) in X12. It was used in many transactions but the most notable use was the 850 (Purchase Order), where it was used in versions up to and including 3060. With version 3070 the NTE segment was removed from the 850. In other transactions the NTE continues to be allowed, but it's position is now fixed in the transaction set. IMPLEMENTATION WTX Pack for X12 EDI Currently supports 8 version of X12 which allow the NTE component to float. Below is an example of the implementation of NTE within these older 850 versions. The NTE segment is defined as a physical segment along with all of the other non-floating segments in the SEGMENT partition of the type tree. Then, instead of being used as a component in the Transaction it is defined as floating in the Transaction. Now that cover the cases where the NTE segment could occur before during or after the components in the Transaction, however, the Transaction also included loops. So for each of the loops the NTE could appear in any position there as well so each loop had the NTE defined as a floating component. And for Loops that contain Loops - each child look had NTE defined as a floating Component ...Until we have defined the floating component for every construct which could contain segments. HOW WIDELY USED in X12: In the 2003 version we support Floating Components in the: 810,820,830,832,840,843,846,850,855,856,860,861, 862,863,865,867,869,870 In the 2040 version we support Floating Components in the: 810,820,830,832,840,843,846,850,855,856,860,861, 862,863,865,867 In 3010 version we support Floating Components in the: 810,820,830,832,840,843,846,850,855,856,860,861, 862,863,865,867 In 3020 version we support Floating Components in the: 810,830,832,840,843,846,850,855,856,860,861,862, 863,865,867 In 3030 version we support Floating Component in the: 810,830,832,840,843,846,850,855,856,860,861,862, 863,865,867 In 3040 version we support Floating Component in the: 810,830,832,840,843,850,855,856,860,861,862,863,865 In 3050 version we support Floating Components in the: 810,830,832,840,843,850,855,856,860,861,862,863,865 In 3060 version we support Floating Component in the: 850,860 In 3070 version and above we support Floating Components in: {NONE} DISCUSSION: It is generally accepted that floating components are a 'bad idea' but they do crop up occasionally. Although X12 now has a policy to not allow floating components for newer version they are still allowed in older versions. Alternatives for modelling these are not very appealing. They really cannot be unordered as everything except the NTE segment is ordered. Another alternative is to have a choice at each possible position in the transaction of the ordered component and the NTE. That just is not practical from a modelling perspective and would result in something that could not be mapped to or from without great acrobatics. WTX users often use the Floating Component construct as a short-cut to avoid to avoid complex structures. To be honest they are probably overused in existing user-created trees (outside of the packs) as it is very tempting when your primary focus is validation to say...."I can have this thing appear almost anywhere" - I'll just slap it in as a FC. Where often come choice groups would be more appropriate. The functional down-side of floating components is trying to create them on output. As the floating component has no fixed location you can not map to it. In the case of the NTE, users generally determine a fixed place to output the NTE and add the segment there (as optional so as to not impact input validation) so that the contents on an NTE can be mapped. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Recursive use of DFDL for variable markup - use case
by Steve Hanson 15 Apr '09

15 Apr '09

>From last week's call: 7. Recursive use of DFDL for variable markup Use of a DFDL annotated element/type to describe an initiator, length prefix, terminator, separator, etc. Steve suggested the most important use of "variable markup-like mechanism" in IBM's WTX product is to reference a location earlier in the bit stream where a delimiter value is found. We handle this already by use of a path expression. The additional variable markup mechanism was to avoid proliferation of keywords for various corner cases on initiator, terminator and separator. Eg., what if you want the initiator to be "Name" or "name" only, not "NAME", "nAmE", etc. So case insensitive is not expressive enough. This can always be modeled, just not as an initiator tag. Feeling was to leave out variable markup (other than for prefix lengths) for v1.0, and to propose the minimum set of extra properties that can be used to address the common use cases, but that IBM needed to see whether this satisfied all WTX use cases. (Post-call update. It doesn't, there is a use case from WTX, Steve will mail this out before next call). The use case is from EDI. EDI transactions consist of an initial header segment which defines, among other things, the separator that is used by the data segments that follow. The problem is that EDI transactions may be processed in their entirety, or individual data segments may be processed without the header segment. For the former case, DFDL supports this fine, using an XPath expression to locate the separator, which is defined as an element, the simple type of which enumerates the allowable values, enabling validation. But for the latter case, the XPath expression won't resolve, as there is no header. An explicit dfdl:separator property could be used instead, being a space separated list of all the allowable values - but that then duplicates the separator element enums, leaving a maintenance problem. <xs:element name="header"> <xs:complexType> <xs:sequence dfdl:lengthKind="implicit"> <xs:element name="separator" dfdl:lengthKind="explicit" dfdl:length="1" dfdl:representation="text"> <xs:simpleType> <xs:restriction base="xs:string"> <xs:enum>xxx</xs:enum> <xs:enum>yyy</xs:enum> <xs:enum>aaa</xs:enum> <xs:enum>bbb</xs:enum> </xs:restriction> </xs:simpleType> </xs:element> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="850"> <xs:complexType dfdl:lengthKind="delimited"> <xs:sequence dfdl:lengthKind="implicit" dfdl:separator="../../header/separator"> <xs:element name="one" type="xs:string" /> <xs:element name="two" type="xs:string" /> <xs:element name="three" type="xs:string" /> <xs:element name="four" type="xs:string" /> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="transaction"> <xs:complexType> <xs:sequence dfdl:lengthKind="implicit"> <xs:element ref="header"/> <xs:element name="segment" maxOccurs="unbounded" /> <xs:complexType> <xs:choice> <xs:element ref="800" /> <xs:element ref="810" /> <xs:element ref="850" /> </xs:choice> </xs:complexType> </xs:element> </xs:sequence> </xs:complexType> </xs:element> Note that WTX solves this by pointing at a global separator element by name, instead of to a separator element in the data by path. At runtime, the infoset value of the global element is used, and if it is not set, the enums are used to provide a list of possible values. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848

1 0

Minutes: OGF DFDL Working Group Call, April-08-2009
by Steve Hanson 09 Apr '09

09 Apr '09

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, April-08-2009 Attendees Suman Kalia (IBM) Steve Hanson (IBM) Mike Beckerle (Oco) Apologies Alan Powell (IBM) Dave Glick (drac) 1. Escape Schemes Alan has mailed latest refinement Overall Agreed with the scope of the escape scheme support, ie, support three well-known variants, and not provide overly complex open ended support Annotation structure. Why dfdl:defineEscapeScheme and dfdl:escapeScheme, instead of just dfdl:escapeScheme and an optional name attribute? For consistency with dfdl:defineFormat, dfdl:defineNumberFormat, etc, and it makes it clear that the top level scope of the naming is a peer to dfdl:defineFormat not inside them. Annotation properties. These need a careful review to make sure that they behave in the expected manner. For example, should escape start/end bracketing be at the start/end of the field, or anywhere in the field? Action raised to review in detail for next call. 2. Validation ranges Need to decide whether to allow restricted use of xs:union to model this. Agreed that this should be supported. For such a union: - The member types must all be derived from the same schema simple type - Any DFDL annotations on member types are a schema definition error Will be added to draft 0.34. 3. Specialized annotations Need to decide whether to drop specialized annotations altogether, or use the scheme below, which does not affect scoping in any way but which makes it clear what is allowed where. Specialized annotations on xsd objects only, dfdl:format on scoping constructs only dfdl;defineFormat => dfdl:format xs:complexType => dfdl:format xs:sequence => dfdl:sequence xs:choice => dfdl:choice xs:group ref => dfdl:group xs:element or xs:element ref => dfdl:element xs:any => dfdl:any xs:simpleType => dfdl:simpleType dfdl:format is exactly as specified in draft 0.33, its properties apply to all relevant objects Scoping rules as specified in draft 0.33 Agreed that this scheme provided the best balance between simplicity and validation capability. Will be added to draft 0.34. 4. Exclusion lists. XML Schema only allows inclusion list of enumerations, does DFDL need to support exclusion list of enumerations? It would be nice if it did, but if we provided a DFDL property that said 'treat enums as exclusion instead of inclusion', removal of the DFDL annotations would change the validation semantic. Agreed that exclusion provision is something that DFDL would inherit from XML Schema, when and if it gets added there. 5. Consuming extraneous data that occurs at the end of the stream This is where the DFDL model matches input data ok, except that there is some extra data in the stream. This can be explicitly modelled, using a hidden optional element. Agreed that whether such a hidden optional element is needed, or whether the data is simply ignored, is up to individual DFDL implementations. The spec will not take a position. 6. 'Floating' definitions A known element, the position of which can be anywhere in a sequence of other elements - is this something DFDL needs to support? Capability is offered by IBM's WTX product. Can be used for comments, but DFDL plans to handle comments post 1.0 using an explicit mechanism or using layering. Real purpose of floating component is for older EDI formats where there is a segment that can appear anywhere, and can appear any number of times. Action raised for IBM to provide a concrete example for discussion. The issue for DFDL is how does a floating component appear in the DFDL infoset, and how does it validate in the sequence. One possibility is a property dfdl:floating=yes/no and if an element has that property set, it can be expected anywhere when parsing, but appears in the correct point in the sequence in the parsed infoset, On unparsing it must appear in in the correct point in the sequence, and is output in that place. 7. Recursive use of DFDL for variable markup Use of a DFDL annotated element/type to describe an initiator, length prefix, terminator, separator, etc. Steve suggested the most important use of "variable markup-like mechanism" in IBM's WTX product is to reference a location earlier in the bit stream where a delimiter value is found. We handle this already by use of a path expression. The additional variable markup mechanism was to avoid proliferation of keywords for various corner cases on initiator, terminator and separator. Eg., what if you want the initiator to be "Name" or "name" only, not "NAME", "nAmE", etc. So case insensitive is not expressive enough. This can always be modeled, just not as an initiator tag. Feeling was to leave out variable markup (other than for prefix lengths) for v1.0, and to propose the minimum set of extra properties that can be used to address the common use cases, but that IBM needed to see whether this satisfied all WTX use cases. (Post-call update. It doesn't, there is a use case from WTX, Steve will mail this out before next call). Actions updated below. Next call 15 April 14:00 UK Meeting closed, 15:05 Actions raised at this meeting No Action 035 AP: Add validation ranges to spec, update specialized annotations in spec. 08/04: Raised. For draft 0.34 036 SH: Provide use case for floating component in a sequence 08/04: Raised Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 04/02: changed from locale to specific properties 18/2: Need more investigation of ICU strict/lax behaviour. 08/04: Not discussed 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 11/02: proposal largely agreed. SH to make minor changes 18/02: AP to document unsigned type behaviour 25/02: no progress 08/04: Not discussed 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 04/02 Call arranged for Friday 11/02: Call took place. Identified useful changes. Consolidate with previous list. 04/03: decided to stay on Schema 1.0. 08/04: Not discussed 024 String XML type 08/04: Not discussed 025 Escape schemes 21/1: discussed requirements 04/02: AP/SH to describe behaviour for known length text fields. Need to discuss if comment escapes should be supported. 11/02 new draft distributed: 18/02: SH up document concerns 25/02: SH and AP have refined proposal ready for approval. 04/03: SH and AP have further refined proposal. 11/03: discussed. suggested a simplified proposal be evaluated. 18/03: SH and AP had further discussions on simplified proposal 08/04: See minutes, review in detail for next call 026 SH: Envelopes and Payloads 08/04: Not discussed explicity, but recursive use of DFDL is tied up with this 027 Property precedence tables 08/04: Not discussed 028 SH: Variable markup 08/04: Discussed briefly at end of call, IBM to see whether there any use cases that require recursive use of DFDL. 029 valueCalc (output length calculation) 08/04: Not discussed 032 DG: Investigate compatibility between DFDL infoset and XDM 08/04: No update 033 AP/TK: Assert/Discriminator semantics. AP to document. TK to check uses of discriminator besides choice. 08/04: In progress within IBM 034 AP: Remove redundant properties, correct old examples 08/04: No update Closed actions: 031 DG: Review dfdl v033 11/02: Initial comments received 18/02: Will include work items 5 and 12. 11/03: complete Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work 026 Remove duration Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL WG Call 2009-04-08: Agenda
by Steve Hanson 08 Apr '09

08 Apr '09

Agenda: Escape Schemes - Alan has mailed latest refinement Validation ranges - need to decide whether to allow restricted use of xs:union to model this ** Specialized annotations - need to decide whether to ** a) drop specialized annotations altogether, b) use the scheme below, which does not affect scoping in any way but which makes it clear what is allowed where Specialized annotations on xsd objects only, dfdl:format on scoping constructs only dfdl;defineFormat => dfdl:format xs:complexType => dfdl:format xs:sequence => dfdl:sequence xs:choice => dfdl:choice xs:group ref => dfdl:group xs:element or xs:element ref => dfdl:element xs:any => dfdl:any xs:simpleType => dfdl:simpleType Exclusion lists - XML Schema only allows inclusion list of enumerations, does DFDL need to support exclusion list of enumerations Consuming extraneous data that occurs at the end of the stream 'Floating' definitions - a known element the position of which can be anywhere in a sequence of other elements - is this something DFDL needs to support Actions ** If you can't make the call today, a vote by e-mail is helpful. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes of OGF DFDL Working Group Call, April-1-2009
by Alan Powell 02 Apr '09

02 Apr '09

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, April-1-2009 Attendees Alan Powell (IBM) Steve Hanson (IBM) Apologies Mike Beckerle(Oco) Dave Glick (drac) Suman Kalia (IBM) As only SH and AP were available the call was cancelled Next call 8 April 14:00 UK Meeting closed, 14:00 Actions raised at this meeting No Action Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 04/02: changed from locale to specific properties 18/2: Need more investigation of ICU strict/lax behaviour. 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 11/02: proposal largely agreed. SH to make minor changes 18/02: AP to document unsigned type behaviour 25/02: no progress 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 04/02 Call arranged for Friday 11/02: Call took place. Identified useful changes. Consolidate with previous list. 04/03: decided to stay on Schema 1.0. 024 String XML type 025 Escape schemes 21/1: discussed requirements 04/02: AP/SH to describe behaviour for known length text fields. Need to discuss if comment escapes should be supported. 11/02 new draft distributed: 18/02: SH up document concerns 25/02: SH and AP have refined proposal ready for approval. 04/03: SH and AP have further refined proposal. 11/03: discussed. suggested a simplified proposal be evaluated. 18/03: SH and AP had further discussions on simplified proposal 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) 031 DG: Review dfdl v033 11/02: Initial comments received 18/02: Will include work items 5 and 12. 11/03: complete 032 DG: Investigate compatibility between DFDL infoset and XDM 033 04/03: Assert/Discriminator semantics. AP to document. TK to check uses of discriminator besides choice. 034 04/03: AP remove redundant properties Closed actions: 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work 026 Remove duration complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: Agenda for OGF WG call 1 April March 2009
by Alan Powell 01 Apr '09

01 Apr '09

NOTE: UK has moved to daylight saving time so the UK and US east coast are 5 hours apart again. Call is at 14:00 UK and 9:00 US east coast. Suggested agenda 1. Validation ranges and exclusion lists - vote 2. dfdl format specializations 3: Variable markup 4. Escape Schemes 5. Actions 6. 'Floating' definitions 7 Consuming extraneous data after schema has be used. 8 Schedule to finish DFDL v1 Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0