Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, April 21-2010 Attendees Steve Hanson (IBM) Alan Powell (IBM) Mike Beckerle (Oco) Stephanie Fetzer (IBM) Tim Kimber(IBM) Apologies Steve Marting (Progeny) Suman Kalia (IBM) 1. DFDL Specification status There have been no public comments so far but this is not unexpected. Steve H is pursuing a press release from IBM and OCO. Mike will send updated information to the OMG. Alan will pursue contacts from the Athena EU project. 2 Nils, defaults and unparsing Steve H, Tim and Alan have had further discussions and concluded and new property was not required so propose - Elements that are zero length and elements that are 'missing' from the data stream should be treated the same. - missingValueInitiatorPolicy should be renamed to emptyValueInitiatorPolicy to distinguish from a missing infoset value on unparsing - Any 'initiator' policy needs to also include terminators so the enumerations should be extended to "none", "initiator", "terminator", "both". and the properties renamed emptyValueDelimiterPolicy and nilValueDelimiterPolicy . The tables have been reworked Nils and Defaults on Parsing initiator region content region Logical Value matching content content matches nilValue nilValueDelimiterPolicy must be initiator or both nil Content does not match nilValue parse content to get value matching empty empty string is a nil value nilValueDelimiterPolicy must be initiator or both nil A default is specified emptyValueDelimiterPolicy must be initiator or both default value (may be nil, empty space or value) empty content content matches nilValue nilValueDelimiterPolicy must be none nil Content does not match nilValue Initiator must not be specified parse content to get value Initiator is specified ERROR empty empty empty string is a nil value nilValueDelimiterPolicy must be none nil A default is specified emptyValueDelimiterPolicy must be none default value (may be nil, empty space or value) Note. The default value can be the empty space. Nils and Defaults on Unparsing Logical Value initiator region contains content region contains Nil (implies nillable) nilValueDelimiterPolicy none empty representation of nil nilValueDelimiterPolicy initiator or both initiator string "" (empty string) (type is xs:string or xs:hexBinary) emptyValueDelimiterPolicy none empty empty emptyValueDelimiterPolicy initiator or both initiator string missing useNilForDefault nilValueDelimiterPolicy none empty representation of nil nilValueDelimiterPolicy initiator or both initiator string default is empty string (type is xs:string or xs:hexBinary) emptyValueDelimiterPolicy none empty empty emptyValueDelimiterPolicy initiator or both initiator string Other default initiator string representation of the default value. a non-nil non-empty-string value initiator string representation of the logical value The above tables (with minor changes) were agreed. The tables currently apply to simpleTypes, examples are needed to confirm that they are correct for complexTypes as well. Steve questioned whether complexTypes can be nillable as WTX allow this. Stephanie will investigate. Subsequently Stephanie confirmed that this was only needed for XML support in WTX so is not needed in DFDL. The Nils, defaulting section in the spec will be reviewed/rewritten to incorporate the new understanding. 3. Current Actions: Updated below. 4 Expressions during Unparsing. In order to reduce complexity and potential performance problems it is proposed to - nilValue and occursStopValue should NOT be able to take an expression. Agreed - Very few properties should be allowed to have forward references. Table below agreed Here's the list of properties that can take an expression, along with some suggested restrictions on forward referencing: byteOrder backward reference only encoding backward reference only initiator backward reference only terminator backward reference only separator backward reference only outputNewline backward reference only length simple elements : backward reference only. complex elements : forward reference allowed Quote "When unparsing with dfdl:lengthKind="explicit", the calculation of dfdl:representationLength() returns the value of the dfdl:length property." A simple element's length needs to be known in advance, else it cannot be parsed. A complex element's length might get resolved while parsing its content, so a forward reference within the scope of the complex element is possible. Having said that, we might want to impose a restriction in IBM's DFDL processor v1.0 that only backward references are allowed. escapeCharacter backward reference only escapeEscapeCharacter backward reference only textNumberDecimalSeparator backward reference only textNumberGroupSeparator backward reference only textNumberExponentCharacter backward reference only binaryFloatRep backward reference only textBooleanTrueRep backward reference only textBooleanFalseRep backward reference only nilValue backward reference only Not clear why this needs to be specified as an expression. ( may already be on a list of open issues somewhere ) nilIndicatorPath backward reference only occursCount backward reference only occursStopValue Not clear why this needs to be specified as an expression. [inputValueCalc] [Parser only] Backward reference only outputValueCalc Forward or backward reference Meeting closed, 14:10 Next call Wednesday 28 April January 2010 13:00 UK (8:00 ET) Next action: 091 Actions raised at this meeting No Action Current Actions: No Action 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update ... 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 24/03: No progress 03/03: Discussions have been taking place on the subset of tests that will be provided. 10/03: work is progressing 17/03: work is progressing 31/03: work is progressing 14/04: And XML test case format has been defined and is being tested. 21/04. Schema for TDML defined. Need to define how this and the test cases will be made public 084 Check behaviour of dfdl:inputValueCalc and outputValueCalc. 14/04: no prorgess 21/04: no progress 085 ALL: publicize Public comments phase to ensure a good review.. 14/04: see minutes 21/04: Press release, OMG and other standards bodies. 086 AP: Nils and Defaults during unparsing - update table 31/03: TK to documetn use cases for parsing 14/04: Investigate new property to control empty string behaviour. 21/04: After investigation a new property is not required. New rules developed and tables updated. Need examples of complexTypes to confirm tables apply. Review Nils, defaulting spec section. 088 define semantics of choiceKind 'fixedLength' 31/03: TK to provide definition of calculable length. Investigate PL/I varchars and Cobol occurs dependingon. 14/04Tim had distributed a document starting the definition of calculable length for the longest choice member. Alan had done some investigation of COBOL occurs depending on and when used in the working section of a program then the maximum storage was reserved but when used in the linkage section the dependent number was uses. We need to understand how the WMB COBOL importer deals with ODO. 21/04: Need to define 'calculable length' and WMB importer ODO behaviour. 089 Need to be able to define unsigned decimal and integer 31/03: MB to agree proposed solution 14/04: MB to agree proposed solution 21/04: Agreed. Closed 090 Semantics of separatorPolicy 14/04: Discussion on detecting when complex types should be defaulted. Examples needed. 21/04: Complex defaulting examples needed for new rules Closed actions No Action Work items: No Item target version status 005 Improvements on property descriptions not started 012 Reordering the properties discussion: move representation earlier, improve flow of topics not started 036 Update dfdl schema with change properties ongoing 042 Mapping of the DFDL infoset to XDM none not required for V1 specification 070 Write DFDL primer 071 Write test cases. 083 Implement RFC2116 097 Remove functions that returns duration 098 occursCountKind is parsing only 099 nilValue and OccuresStopValue cannot have an expression. On unparsing only outputValueCalc can have a forward reference. Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell@uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU