Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, February 17-2010 Attendees Steve Hanson (IBM) Alan Powell (IBM) Steve Marting (Progeny) Peter Lambros (IBM) Stephanie Fetzer (IBM) Apologies Mike Beckerle (Oco) Suman Kalia (IBM) Tim Kimber(IBM) 1. Comments of latest discriminators doc v5. Steve questioned: It is a processing error if none of the choice branches are known to exist Steve: I know we agreed on this but I?m concerned that I am now unable to model the situation where I have a header, a footer, and in between either nothing or exactly 1 of a choice of n records It was agreed that the body could be an optional complex element with the choice as its content. Also confirmed dfdl:discriminator can be an annotation on · a local xs:element declaration · an xs:element reference · an xs:group reference (when the top level of a choice branch) · an xs:sequence (when the top level of a choice branch) · an xs:choice (when the top level of a choice branch) Action 045 will be closed. 2. Remaining 037 review issues 16.2 scannablility with lengthKind pattern: Confirm that this is what we agreed In summary, you can use a data pattern on any element (complex, simple text, simple binary) as long as the bytes are legal in the stated encoding, which where binary data is involved in practice means an 8-bit ASCII encoding. By 8-bit ASCII we really mean an encoding where all the codepoints from 0-255 map to the equivalent value. Subsequent investigation indicates that 'all' 8-bit ASCII encodings have gaps so there isn't a valid character. Mike has suggested 1) for all ascii-based character sets, we say that bytes 0x00 to 0xFF all map to exactly those codepoints in ISO 10646 for the infoset, and vice versa. 2) define dfdl:encoding="bytes" as a special character set name which has the above property. Briefly discussed but no conclusion. 3. Go through Actions Updated below 4. Biztalk comparison Short discussion which noted that DFDL could match Biztalk representation properties but was missing support for attributes and some simple types. Decided that we don't need to do anything in DFDL v1. 5 DFDL v1 Specification completion. It was agreed that the spec should be in the OGF review process by OGF 28 in March. Alan stated it needed 2 more drafts. Draft 39 with all the updates completed for a final WG review - available 24 Feb Comments by 3 March Draft 40 with updates for OGF submission - available 5 March Meeting closed, 14:00 Next call Wednesday 24 February January 2010 13:00 UK Next action: 083 Actions raised at this meeting No Action Current Actions: No Action 049 20/05 AP Built-in specification description and schemas 03/06: not discussed 24/06: No Progress 24/06: No Progress (hope to get these from test cases) 15/07: No progress. Once available, the examples in the spec should use the dfdl:defineFormat annotations they provide. ... 14/10: no progress 21/10: Discussed the real need for this being in the specification. It seemed that the main value is it define a schema location for downloading 'known' defaults from the web. 28/10: no progress 04/11: no progress 11/11: no update 18/11: no update 25/11: Agreed to try to produce for CSV and fixed formats 04/12: no update 09/12: no update 16/12: no update 23/12: no update 06/01: no progress. If there is no resource to complete this action it can be deferred 13/01:no progress 20/01: no progress 27/01: no progress 29/01: No progress. The predefined formats do not need to be available when the spec is published. Suman said that he had been mapping COBOL structures to DFDL and it didn't look as though the way text numbers are define is very usable. He will document for next call 03/02: No progress 10/02: No progress 17/03: No progress 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 079 MB:Encoding for binary fields when lenghtkind is pattern 17/02: Discussed but no conclusion 080 AP:Clarify semantics of fn:poisition and fn:count 17/02: no progress 081 AP: Inf and Nan The description is the way ICU behaves but need clarification. It isn't clear how inf and Nan are represented in the infoset. Need to investigate if XML allows these values 17/02: XML allows Nan and inf for float and double Dfdl will do the same. Requires more investigation of ICU. Closed actions No Action 045 20/05 AP: Speculative Parsing 27/05: Psuedo code has been circulated. Review for next call 03/06: Comments received and will be incorporated 09/06: Progress but not discussed 17/06: Discussed briefly 24/06: No Progress 01/07: No Progress 15/07: No progress. MB not happy with the way the algorithm is documented, need to find a better way. 29/07: No Progress 05/08: No Progress. Will document behaviour as a set of rules. 12/08: No Progress ... 16/09: no progress 30/09: AP distributed proposal and others commented. Brief discussion AP to incorporate update and reissue 07/10: Updated proposal was discussed.Comments will be incorporated into the next version. 14/10: Alan to update proposal to include array scenario where minOccurs > 0 21/10: Updated proposal reviewed 28/10: Updated proposal reviewed see minutes 04/11: Discussed semantics of disciminators on arrays. MB to produce examples 11/11: Absorbing action 033 into 045. Maybe decorated discrminator kinds are needed after all. MB and SF to continue with examples. 18/11: Went through WTX implementation of example. SF to gather more documentation about WTX discriminator rules. 25/11: Further discussion. Will get more WTX documentation. Need to confirm that no changes need to Resolving Uncertainty doc. 04/11: Further discussion about arrays. 09/12: Reviewed proposed discriminator semantic. 16/12: Reviewed discriminator examples and WTX semantic. 23/12: SF to provide better description of WTX behaviour and invite B Connolley to next call 06/01:B Connolly not available. SF to provide more complete description. 13/01: Stephaine took us through a description of WTX identifiers. Mike agreed to write up in DFDL terms. 20/01: Mike will write up 27/01: further discussion of discriminators 29/01: Alan had emailed both proposals but not enough time to discuss 02/02: Agreed to adopt 'component exists' semantics for discriminators 10/02: 'component exists' proposal updated. comments by next call. 17/02: reviewed needs minor updates. Closed 082 MB: Should alignment be 0 or 1 based 17/02: Mike has reworded the section to be 1 based and Agreed this was OK Closed Work items: No Item target version status 005 Improvements on property descriptions not started 012 Reordering the properties discussion: move representation earlier, improve flow of topics not started 036 Update dfdl schema with change properties ongoing 042 Mapping of the DFDL infoset to XDM none not required for V1 specification 069 ICU fractional seconds 039 070 Write DFDL primer 071 Write test cases. 072 it is a processing error if the number of occurrences in the data does not match the value of the expression or prefix 039 073 Rename dfdl:separatorPolicy="required" to "always". 039 Defferred untilaction 071 agreed 078 document UPA checks 039 079 Semantics of length=0, nil handling and defaults. (A071) 039 080 Tlog: Allow LengthKind delimited for packed/bcd (A074) 039 081 Update empty sequence section (A075) 039 082 semantics of minOccurs= 0 on choice branches (A076) 039 083 Implement RFC2116 084 Length|Kind pattern scanability rules 085 Invalid character substitution 086 infoset round tripi: Rephrase sentence 'It is possible to define a schema so that when infoset unparsed and the datastream reparsed, the same infoset will be produced' 087 Clarify use of relative paths in global components. 088 'DFDL expression' 089 Ageed that dfdl:represetnation 'text' is implied for strings and dfdl:represetnation 'binary' is implied for hexbinary 091 textStringPadCharacter textNumberPadCharacter must be a 1 byte character if the char set encoding is variable width? 092 finalDocumentTerminatorCanBeMissing and finalDocumentSeparatorCanBeMissing allowed only in 'default' format 093 remove textNumberFormat and textCalendarFormat. 094 Alignment should be 1 based Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell@uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU