- dfdl-wg - lists.ogf.org

Numbers with multiple ranges
by Steve Hanson 11 Mar '09

11 Mar '09

One of the existing non-XML parsers from IBM is capable of modelling an integer or decimal with non-contiguous ranges, eg, 0-100, 200-300, 400-500. It is also possible to model this using XML Schema, using a user-defined simple type which is a union of simple type restrictions. However it is not possible to model this in DFDL because unions are not supported. A choice could be used, but that would give different named elements in the resultant infoset depending on the value, which is not ideal. I believe this is use case that we should consider for DFDL 1.0. We could decide to allow unions, but with a constraint that the members of the union had to be restrictions of the same built-in type, and that clashing DFDL properties on the members was an error. I believe the existing scoping rules for visiting types would still apply ok. No annotation appears on the xs:union only on the xs:simpleType as currently. The net effect is that our parsing rules are the same as currently, but we gain validation capability. I am not in favour of more widespread support for unions. I don't have a compelling use case for union of different logical types, eg, xs:dateTime and xs:string. Your thoughts are welcome. !-- integer range 0 - 100 <xsd:simpleType name="intRange1"> <xsd:restriction base="xsd:int"> <xsd:maxInclusive value="100"/> <xsd:minInclusive value="0"/> </xsd:restriction> </xsd:simpleType> !-- integer range 200 - 300 <xsd:simpleType name="intRange2"> <xsd:restriction base="xsd:int"> <xsd:maxInclusive value="300"/> <xsd:minInclusive value="200"/> </xsd:restriction> </xsd:simpleType> !-- integer range 400-500 <xsd:simpleType name="intRange3"> <xsd:restriction base="xsd:int"> <xsd:maxInclusive value="500"/> <xsd:minInclusive value="400"/> </xsd:restriction> </xsd:simpleType> !-- Union of above types <xsd:simpleType name="intRange"> <xsd:union memberTypes="intRange1 intRange2 intRange3"/> </xsd:simpleType> !-- Union of anonymous types being restrictions of above types <xsd:simpleType name="intRangeLocal"> <xsd:union> <xsd:simpleType> <xsd:restriction base="intRange1"/> </xsd:simpleType> <xsd:simpleType> <xsd:restriction base="intRange2"/> </xsd:simpleType> <xsd:simpleType> <xsd:restriction base="intRange3"/> </xsd:simpleType> </xsd:union> </xsd:simpleType> !-- Union of anonymous types being local restrictions <xsd:simpleType name="intRangeLocalRestrictions"> <xsd:union> <xsd:simpleType> <xsd:restriction base="xsd:int"> <xsd:minInclusive value="0"/> <xsd:maxInclusive value="100"/> </xsd:restriction> </xsd:simpleType> <xsd:simpleType> <xsd:restriction base="xsd:int"> <xsd:minInclusive value="200"/> <xsd:maxInclusive value="300"/> </xsd:restriction> </xsd:simpleType> <xsd:simpleType> <xsd:restriction base="xsd:int"> <xsd:minInclusive value="400"/> <xsd:maxInclusive value="500"/> </xsd:restriction> </xsd:simpleType> </xsd:union> </xsd:simpleType> Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

4 6

Errors in Running VirtualXML Example
by Jyotishman Pathak 10 Mar '09

10 Mar '09

Hello all, I have been trying to play with virtualxml sample examples, and in particular, the example about Binary Data (cionverting the byte sequence as XML data). Now, if I try to run the example usign the "some-binary.data" file provided with the VirtualXML release, everything works fine. However, if I try to generate this data file using the C program ("some-binary.c") provided with the release, I get the following exception: Exception in thread "main" java.lang.RuntimeException: com.ibm.xml.xq.virtual.dfdl.DFDLException: XX0000: unknown (Could not complete contents.) at com.ibm.xml.xq.virtual.dfdl.DFDLInputMediator.buildChild(DFDLInputMediator.java:473) at com.ibm.xml.xq.virtual.dfdl.DFDLInputMediator.buildNext(DFDLInputMediator.java:154) at com.ibm.xml.xq.dm.cache.CacheManager.getCachedFollowingSibling(CacheManager.java:403) at com.ibm.xml.xq.dm.cache.dom.DOMCachedNode.getCachedFollowingSibling(DOMCachedNode.java:235) at com.ibm.xml.xq.dm.cache.CacheFocus.toNext(CacheFocus.java:420) at com.ibm.xml.xq.dm.util.Copier.copySequence2Content(Copier.java:209) at com.ibm.xml.xq.dm.util.Copier.copyContents(Copier.java:279) at com.ibm.xml.xq.dm.util.Copier.copySequence2Content(Copier.java:214) at com.ibm.xml.xq.dm.util.Copier.copyContents(Copier.java:279) at com.ibm.xml.xq.dm.util.Copier.copySequence2Content(Copier.java:206) at com.ibm.xml.xq.dm.util.Copier.copy(Copier.java:119) at com.ibm.xml.xq.dm.util.Copier.copy(Copier.java:71) at com.ibm.xml.xq.processor.Run.output(Run.java:442) at com.ibm.xml.xq.processor.Run.run(Run.java:408) at com.ibm.xml.xq.virtual.Virtual.main(Virtual.java:52) Caused by: com.ibm.xml.xq.virtual.dfdl.DFDLException: XX0000: unknown (Could not complete contents.) at com.ibm.xml.xq.virtual.dfdl.DFDLInputMediator.buildChild(DFDLInputMediator.java:469) ... 14 more I have tried executing this example in both windows and linux platforms, but with no success. Could you please assist me? Thanks, - Jyoti

2 1

Agenda for OGF WG call 11 March 2009
by Alan Powell 10 Mar '09

10 Mar '09

Suggested agenda 1. dfdl format specializations 2. Escape Schemes 3. Consuming extraneous data after schema has be used. 4. 'Floating' definitions 5. Validation ranges and exclusion lists 6 Actions 7 AOB Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: dfdl:sequence and other specialized annotations
by Steve Hanson 09 Mar '09

09 Mar '09

Although it was minuted in last week's call that we decided to drop specialised annotations, I don't actually recall that was decision on the call. The proposal below is in accordance with the suggestion by Martin, and echoed by Dave, that we have only specialised annotations for xsd objects. Schema object DFDL annotation allowed Notes Schema declaration dfdl:format Only within an enclosing dfd:defineFormat. Element declaration dfdl:element Just those properties allowed on elements Element reference dfdl:element Just those properties allowed on elements. Overrides dfdl:element properties on global element Complex type definition dfdl:complexType Full dfdl:format properties. For scoping purposes only. Simple type definition dfdl:simpleType Just those properties allowed on simple types. Sequence declaration dfdl:sequence Just those properties allowed on sequences. Choice declaration dfdl:choice Just those properties allowed on choices. Group reference dfdl:group Just those properties allowed on sequence/choice. Overrides dfdl:sequence/dfdl:choice properties on global group sequence/choice model group An alternative as I previously wrote is to allow dfdl:sequence and dfdl:choice on a group reference. Group reference (to sequence group) dfdl:sequence Just those properties allowed on sequence/choice. Overrides dfdl:sequence properties on global group model sequence group Group reference (to choice group) dfdl:choice Just those properties allowed on sequence/choice. Overrides dfdl:choice properties on global group model choice group Although this does not look as good visually, it is worth bearing in mind section 22.2 of the spec: 22.2 Group Reference A copy of the definition referenced by a group reference is substituted inline for its declaration while combining any DFDL format annotation elements associated with it as given in Section ‎10.5. (TBD: xref to scoping?) Processing then proceeds as for an anonymous group declaration. Note: The use of specialised annotations relies on the removal of something that we had talked about but not yet reflected in the spec - dfdl:format on global xsd:group definitions. The idea of allowing this came about fairly recently when we changed the rules for hidden objects. I was unhappy that embedded hidden xsd objects within DFDL annotations would make them invisible to a schema validator, so dfdl:hidden was changed to be a reference to a global group containing the hidden objects. I suggested that we could also put scoping annotations on global groups to support scoping within the hidden construct, it seemed a good idea at the time, but it was not fully explored. Thinking it through, because the global group effectively disappears when the reference is resolved, then so does its annotations. I'm therefore happy to retract scoping annotations on global groups if it makes specialised annotations work. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 From: Steve Hanson/UK/IBM@IBMGB To: Suman Kalia <kalia(a)ca.ibm.com> Cc: dfdl-wg(a)ogf.org Date: 02/03/2009 23:45 Subject: Re: [DFDL-WG] Reducing the number of DFDL properties. Suman A DFDL annotation on a group reference overrides a DFDL annotation on the content of the model group (ie, sequence, choice), not a DFDL annotation on the global group. It allows you to override the sequence/choice properties at point of use. The DFDL annotation on a global group is like that on a complex type - it is a scoping construct only. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Suman Kalia <kalia(a)ca.ibm.com> 02/03/2009 22:02 To Steve Hanson/UK/IBM@IBMGB cc dfdl-wg(a)ogf.org, "Dave Glick" <dglick(a)dracorp.com> Subject Re: [DFDL-WG] Reducing the number of DFDL properties. I can see where you are coming from and I agree with most of your proposal except putting dfdl:sequence, dfdl:choice , dfdl:all on group references as it does not give the right optics.. From consistency point of view, the references (group and element references) should have the equivalent/same set of annotations that can appear on the referenced artifacts. Suman Kalia IBM Toronto Lab WMB Toolkit Architect and Development Lead WebSphere Business Integration Application Connectivity Tools http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h… Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com From: Steve Hanson <smh(a)uk.ibm.com> To: Suman Kalia/Toronto/IBM@IBMCA Cc: dfdl-wg(a)ogf.org, "Dave Glick" <dglick(a)dracorp.com> Date: 03/02/2009 12:45 PM Subject: Re: [DFDL-WG] Reducing the number of DFDL properties. If I recall, one reason for having the specialised elements was so that the content of the annotations could be more tightly validated by the schema-for-dfdl xsd. However, the problem with the specialised annotations is that we do not provide them for all circumstances. For example, the spec allows me to use dfdl:sequence on an xs:sequence but not an xs:group reference to a global xs:group with compostion xs:sequence. Nor does it allow me to use a specialised element on xs:simpleType or xs:restriction. So as it stands, validation would not be fool-proof anyway. I think we must either ditch specialisations, or mandate the use of specialisations on the corresponding schema objects. The latter is more workable now we have changed the scoping rules so that scoping of dfdl properties can only be done from complex type or schema level, as below. But it does mean that dfdl annotations are less flexible. dfdl;defineFormat => dfdl:format xs:complexType => dfdl:format xs:group => dfdl:format xs:sequence or xs:group ref to a global group with xs:sequence => dfdl:sequence xs:choice or xs:group ref to a global group with xs:choice => dfdl:choice xs:element or xs:element ref => dfdl:element xs:any => dfdl:any xs:simpleType, xs:restriction => dfdl:simpleType In other words, you use dfdl:format when you are scoping properties, you must use specialised annotation for specific objects. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Suman Kalia <kalia(a)ca.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 02/03/2009 16:25 To "Dave Glick" <dglick(a)dracorp.com> cc dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Subject Re: [DFDL-WG] Reducing the number of DFDL properties. I am not sure what kind of confusion or redundancy is caused by these specialized annotations. In the absence of these specialized annotations, you will have to go through plethora of annotations and determine which ones are applicable for sequence , choice, all, elements etc.. The danger is we don't have these levels of abstractions, then number of folks (specifically implementers) would build them anyway and then we will have to contend with incompatible abstractions.. Suman Kalia IBM Toronto Lab WMB Toolkit Architect and Development Lead WebSphere Business Integration Application Connectivity Tools http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h… Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com From: "Dave Glick" <dglick(a)dracorp.com> To: "Alan Powell" <alan_powell(a)uk.ibm.com>, <mbeckerle.dfdl(a)gmail.com> Cc: dfdl-wg(a)ogf.org Date: 03/02/2009 11:16 AM Subject: Re: [DFDL-WG] Reducing the number of DFDL properties. Alan/Mike, I agree with this – one of my complaints with v033 was that these specialized annotation elements just added confusion and redundancy. Dave --- David Glick | dglick(a)dracorp.com | 703.299.0700 x212 Data Research and Analysis Corp. | www.dracorp.com From: dfdl-wg-bounces(a)ogf.org [mailto:dfdl-wg-bounces@ogf.org] On Behalf Of Alan Powell Sent: Monday, March 02, 2009 11:03 AM To: mbeckerle.dfdl(a)gmail.com Cc: dfdl-wg(a)ogf.org Subject: Re: [DFDL-WG] Reducing the number of DFDL properties. Mike Thanks Can I also suggest dropping the dfdl:sequence, dfdl:choice, dfdl:element and dfdl:any specialized annotations and just have dfdl:format. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 From: "Mike Beckerle" <mbeckerle.dfdl(a)gmail.com> To: Alan Powell/UK/IBM@IBMGB Cc: <dfdl-wg(a)ogf.org> Date: 28/02/2009 19:38 Subject: RE: Reducing the number of DFDL properties. If a property is redundant, I'm in favor of dropping it. If a property adds generality that we don't have a use case for, I'm in favor of dropping it. I am happy to drop type substitution from v1.0. It's a convenience that can be achieved a different way. E.g., if you really want "float" to mean "float in my particular binary representation", then just put a type definition with DFDL annotations in a different namespace, and when you write your DFDL schema, arrange for the default namespace to pick it up from your namespace, and not xs:float. <xs:element name="myElement" type="float"/>  <xs:element name="anotherElem" type="xs:float"/>  If you want the XSD unadorned "float" type, be explicit and use "xs:float". Voila - no loss of flexibility, equal textual convenience. I think that would satisfy the community that really wanted very compact "slideware acceptable" schemas. This is the same group that wants short-form annotations as well. Mike Beckerle | OGF DFDL WG Co-Chair | CTO | Oco, Inc. Tel: 781-810-2100 | 100 Fifth Ave., 4th Floor, Waltham MA 02451 | mbeckerle.dfdl(a)gmail.com From: Alan Powell [mailto:alan_powell@uk.ibm.com] Sent: Thursday, February 26, 2009 12:22 PM To: mbeckerle.dfdl(a)gmail.com Cc: dfdl-wg(a)ogf.org Subject: Reducing the number of DFDL properties. Mike A number of people at IBM have become concerned at the number of properties in DFDL and have identified a number of 'usability' properties that could be dropped. They feel that we should be simplifying the properties wherever possible and not introducing multiple ways of doing the same function without very good reason. The following are offered for consideration. 1. lengthKind='nullterminated' This is just shorthand for lengthKind=delimited and terminator='%Null'. It was felt this this is not even the most common terminator so why have a special case? 2. trimKind It is felt that there aren't any cases when you would want to pad but not trim and vice versa so make padKind control both. 3. typeSubstitution. Is this needed in DFDL v1? Can you consider these before the call next week Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

OGF DFDL Working Group weekly call - summer time
by Steve Hanson 09 Mar '09

09 Mar '09

Please be aware that in North America, summer time starts this week which is 3 weeks earlier than in UK. The call is therefore at 10am Eastern time for next 3 weeks. Regards Steve Hanson Programming Model Architect WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, March-04-2009
by Alan Powell 05 Mar '09

05 Mar '09

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, March-04-2009 Attendees Mike Beckerle(Oco) Alan Powell (IBM) Steve Hanson (IBM) Tim Kimber (IBM) Apologies Dave Glick (drac) 1. XML Schema 1.1 It was agreed that the lack of support for Schema 1.1 in tools, libraries etc would be an inhibitor to DFDL implementations so it was agreed to remain on XML schema 1.0 . SH highlighted the main isuues with staying on XSD 1.0 1) xs:all improvements - relaxing min/max occurs, allowing wildcards, I don't see an alternative to our current proposal of dfdl:sequenceKind. 2) weakened wildcard Either live with this restriction, or say that the DFDL parser will implement the 1.1 rule where an optional element takes precedent over a wildcard and say that if you use this feature of DFDL, you should avoid XML schema tools that implement this UPA check. (It is my understanding that many tools that validate XML Schema for correctness do not include the UPA checks). These issues need to be resolved. It was noted thet Schema 1.1 assert only allow reference to children of the item with the assert and that it would be desirable if DFDL was compatible. As there are dfdl use cases were reference to earlier fields is required AP to investigate if assigning earlier fields to variables in sufficient. 2. DFDL;ASSERT and DFDL:DISCRIMINATOR Discussion to clarify the semantics of assert and discriminator. Decided that dfdl:discriminator should only be used to resolve choices. TK to investigate if there are other use cases where discriminator would be useful. dfdl:assert is to ensure that a document is well formed which, while possibly applying similar tests, is separate from validation. Confirmed that assert cannot be turned off. AP to document semantics. 3. Reducing number of properties General agreement to reduce properties when they are redundant and don't support a common use case. LengthKind=NullTerminated, typeSubstitution, and trimkind.will be removed. dfdl:format specialization such as dfd;:sequence were also discussed. It had been pointed out in emails that there are two stable options - Only dfdl:format with no specializations - Only specializations and no general dfdl:format. Supporting both dfdl:format and specialization was felt to be undesirable. SH point out that the second option was not practical for all constructs as complex type definitions which put properties in scope would require all possible properties. I was agreed that for dfdl v1 the specializations would be dropped. 4. AOB To be discussed - Escape Schemes - Consuming extraneous data after schema has be used. - 'Floating' definitions - Validation ranges and exclusion lists - Actions Next call 11 March 14:00 Meeting closed, 15:00 GMT Actions raised at this meeting No Action 033 04/03: Assert/Discriminator semantics. AP to document. TK to check uses of discriminator besides choice. 034 04/03: AP remove redundant properties Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 04/02: changed from locale to specific properties 18/2: Need more investigation of ICU strict/lax behaviour. 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 11/02: proposal largely agreed. SH to make minor changes 18/02: AP to document unsigned type behaviour 25/02: no progress 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 04/02 Call arranged for Friday 11/02: Call took place. Identified useful changes. Consolidate with previous list. 04/03: decided to stay on Schema 1.0. 024 String XML type 025 Escape schemes 21/1: discussed requirements 04/02: AP/SH to describe behaviour for known length text fields. Need to discuss if comment escapes should be supported. 11/02 new draft distributed: 18/02: SH up document concerns 25/02: SH and AP have refined proposal ready for approval. 04/03: SH and AP have further refined proposal. 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) 031 DG: Review dfdl v033 11/02: Initial comments received 18/02: Will include work items 5 and 12. 032 DG: Investigate compatibility between DFDL infoset and XDM 033 04/03: Assert/Discriminator semantics. AP to document. TK to check uses of discriminator besides choice. 034 04/03: AP remove redundant properties Closed actions: 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work 026 Remove duration complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Agenda for OGF WG call 04 March 2009
by Alan Powell 04 Mar '09

04 Mar '09

Suggested agenda 1. XSD 1.1 2. Escape Schemes 3. Assert/Discriminators 4. Dropping properties 5. Consuming extraneous data after schema has be used. 6. 'Floating' definitions 7. Validation ranges and exclusion lists 8 Actions 9 AOB Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Reducing the number of DFDL properties.
by Alan Powell 03 Mar '09

03 Mar '09

Mike A number of people at IBM have become concerned at the number of properties in DFDL and have identified a number of 'usability' properties that could be dropped. They feel that we should be simplifying the properties wherever possible and not introducing multiple ways of doing the same function without very good reason. The following are offered for consideration. lengthKind='nullterminated' This is just shorthand for lengthKind=delimited and terminator='%Null'. It was felt this this is not even the most common terminator so why have a special case? trimKind It is felt that there aren't any cases when you would want to pad but not trim and vice versa so make padKind control both. typeSubstitution. Is this needed in DFDL v1? Can you consider these before the call next week Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

6 12

Fw: Reducing the number of DFDL properties.
by Suman Kalia 27 Feb '09

27 Feb '09

nullTerminated is a very familiar paradigm which the users understand. We need to keep usability in mind; keep simple tasks simple. Yes it is redundant and you can do the same with 2 properties ( lengthKind=delimited and terminator=NULL) but it is well worth to keep it.. Same argument can be made for trimKind.. I am ok if we drop typeSubstitution although it is a convenient way to specify whole bunch properties grouped together for a type and reuse it. Suman Kalia IBM Toronto Lab WMB Toolkit Architect and Development Lead WebSphere Business Integration Application Connectivity Tools http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.h… Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 02/26/2009 01:52 PM ----- From: Alan Powell <alan_powell(a)uk.ibm.com> To: mbeckerle.dfdl(a)gmail.com Cc: dfdl-wg(a)ogf.org Date: 02/26/2009 12:27 PM Subject: [DFDL-WG] Reducing the number of DFDL properties. Mike A number of people at IBM have become concerned at the number of properties in DFDL and have identified a number of 'usability' properties that could be dropped. They feel that we should be simplifying the properties wherever possible and not introducing multiple ways of doing the same function without very good reason. The following are offered for consideration. 1. lengthKind='nullterminated' This is just shorthand for lengthKind=delimited and terminator='%Null'. It was felt this this is not even the most common terminator so why have a special case? 2. trimKind It is felt that there aren't any cases when you would want to pad but not trim and vice versa so make padKind control both. 3. typeSubstitution. Is this needed in DFDL v1? Can you consider these before the call next week Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

2 1

DFDL: Minutes from OGF WG call, February-25-2009
by Alan Powell 26 Feb '09

26 Feb '09

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, February-25-2009, Attendees Alan Powell (IBM) Steve Hanson (IBM) Dave Glick (drac) Tim Kimber (IBM) Apologies Mike Beckerle(Oco) 1. Discussion of D Glick Spec 033 review Dave has done a thorough review of DFDL Spec v033 and circulated an email with the major point which was discussed on this call. Major areas discussed - Balance of formal spec vs Primer. The spec should be make a clearer distiction between normative and non-normative information. - Parts of the document sound like implementation details which shouldn't be included - DG suggested clearer document structure but wasn't sure on the best oreder to list representation properties. Possible suggestion is alphabetic list of properties with a number of indexes giving different views. - Should be better with RFC2119 terminology SHOULD, MUST, etc - Discussed whether dfdl infoset should be should be subset/compatible with XML infoset or XDM. General agreement that it should be as compatible as possible with XDM. DG to investigate hwo close they can be made. - Types of DFDL properties should be more clearly specified. - DG wondered whether case sensitivity would work for all locales. Suggested investigate using collations. 5. AOB IBM want to have the spec technically complete within the next 6-8 weeks. Next call 1 March 14:00 Meeting closed, 15:00 GMT Actions raised at this meeting No Action 032 DG: Investigate compatibility between DFDL infoset and XDM Current Actions: No Action 012 AP/SH: Update decimalCalendarScheme 10/9: Not allocated yet 17/9: No update 24/9: Add calendar binary formats to actions 22/10: No progress 16/1: proposal distributed and discussed. Will be redistributed 21/1: add locale, 04/02: changed from locale to specific properties 18/2: Need more investigation of ICU strict/lax behaviour. 020 SH: Resolve packedDecimalSignCodes behaviour depends on NumberCheckPolicy 22/10: No progress 10/12: added how to decide to overpunch and sign position 11/02: proposal largely agreed. SH to make minor changes 18/02: AP to document unsigned type behaviour 25/02: no progress 023 MB: Review Schema 1.1 29/1: AP and SH to talk to Sandy Gao 04/02 Call arranged for Friday 11/02: Call took place. Identified useful changes. Consolidate with previous list. 024 String XML type 025 Escape schemes 21/1: discussed requirements 04/02: AP/SH to describe behaviour for known length text fields. Need to discuss if comment escapes should be supported. 11/02 new draft distributed: 18/02: SH up document concerns 25/02: SH and AP have refined proposal ready for approval. 026 SH: Envelopes and Payloads 027 Property precedence tables 028 Variable markup 029 valueCalc (output length calculation) 031 DG: Review dfdl v033 11/02: Initial comments received 18/02: Will include work items 5 and 12. 032 DG: Investigate compatibility between DFDL infoset and XDM Closed actions: 034 Work items: No Item 001 String XML type (Ian P) - Apr 30, 2008 002 Escape schemes (Ian P) - Apr 30, 2008 003 Variables - ??, 2008 (Mike) 005 Improvements on property descriptions - ??, 2008 (All - split TBD) 006 Envelopes and Payloads (Steve) - Apr 30, 2008 007 (from draft 32) valueCalc (Mike) - ??, 2008 mostly complete 008 (from draft 32) Property precedence for writing (Steve) - under review 009 (from draft 32) Variable markup (Steve) - Mar 31, 2008 proposal needs writing up 010 (from draft 32) Assertions, discriminators and choice, including discussion of timing option (Suman) - Mar 31, 2008 * in progress * 011 (from draft 32) How speculative parsing works (combining choice and variable-occurence - currently these are separate) ??, 2008 (IBM) in progress 012 (from draft 32) Reordering the properties discussion: move representation earlier, improve flow of topics ??, 2008 (Alan) * not started * 025 Augmented infoset and unparsing (Alan) added but needs work 026 Remove duration complete - specification updated Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0