Open Grid Forum: Data Format Description Language Working Group

OGF DFDL Working Group Call, December-16-2009

Attendees
Mike Beckerle (Oco)
Steve Hanson (IBM)
Alan Powell (IBM)
Steve Marting (Progeny)
Stephanie Fetzer (IBM)
Tim Kimber (IBM)

Apologies
Suman Kalia (IBM)
Peter Lambros (IBM)



1.        045  - Disciminators
We went through the examples of discriminator on resolving choices and optional element that Alan had produced in some detail.

Noted that all the elements on choice branches should have discriminators to resolve the parent choice not just the simple element, otherwise asymmetry between simple and complex choice branches.

Stephanie explained some of the WTX semantics.
- an identifier must be on a mandatory item/group (minOccurs > 0)
- an identifier has an implicit rule ( exists() ) is no other rule is specified.
- identifier is not used on the branches of a choice, but is on a partitioned group.

It was suggested that DFDL should adopt the WTX identifier flag with separate rule model rather then distinct assert rules and discriminator rules.

Stephanie had to leave so discussion postponed until next call

2.        Clarification of postfix separators, terminators,finalTerminatorCanBeMissing
Discussion to clarify where dfdl:documentFinalTerminatorCanBeMissing and dfdl:documentFinalSeparatorCanBeMissing apply.
 It was agreed that they apply on the root element passed to the parser which may not be a global element.

3.        Does the parser/serializer have to start on a global element?
As part of the discussion of 2 above it was decided after all not to introduce a property to designate global root elements as it was felt that the only sensible options would be for the property to be enforced. ie a specified element MUST have the property, or don't have the property..

But need Stephanie and Suman to agree.

4. LengthUnits = 'bits'
Mike will distribute his proposal to mailing list

5. Semantics of length=0
Need to go through each of the lengthKinds to understand the interaction of length=0, nil handling and defaults.
ACTION

6. Go through remaining actions
Updated below

7. Draft 037 review
Please submit comments by Monday 21st Dec

Agree terminology

Agreed that:
-A schema has components which have dfdl annotations
.- Dfdl annotations have properties (same as XSDL)  but need to distinguish between format/representation properties and other properties such as name, ref, selector.

Not discussed.
-The data/data stream/message has elements/fields
-The infoset has items

7a. Byte Order Mark and Unicode signature
Tim stated that the statement that BOM is treated as data is not implementable consistently with ICU. Would prefer it to say that BOM are not allowed.
But his use of BOM included both UTF-16 BOM and UTF-8 unicode signature. Not clear that UTF-16 BOM is a problem.
Tim will investigate further ICU behaviour. ACTION

8. Plan to finish DFDL v1 Schedule
Activity
Schedule
Who
Complete Action items
             - 18 Dec 2009
 WG
Complete Spec Write up work items
            – 23 Dec 2009
AP
Restructure and complete specification
              - 23 Dec 2009
AP
Issue Draft 038
23 Dec 2009
WG review WG review
7 Dec – 08 Jan 2010
WG
Incorporate review comments
4 Jan - 29 Jan 2010
AP +
Issue Draft 039
15 Jan 2010
Incorporate review comments
4 Jan - 29 Jan 2010
AP +
Issue Draft 040
29 Jan 2010
Initial OGF Editor Review Initial Editor review
1 Feb - 1 Mar 2010
OGF
Initial GFSG review
1 Feb - 1 Mar 2010
Issue Draft 041
1 Mar 2010
OGF Public Comment period (60 days)
1 Mar - 30 Apr 2010
OGF
OGF 28 Munich
15-19 March 2010
Incorporate comments Incorporate comments
28 May 2010
Issue Draft 042
28 May 2010
Final OGF Editor Review Final  Editor review
June  2010
OGF
final GFSG review
June  2010
Issue Final specification
30 June 2010
Publish proposed recommendation
1 July 2010
Grid recommendation process
1 Jan - 1 April 2011




Meeting closed, 15:00

Next call 23 December 13:00 UK

Next action: 073

Actions raised at this meeting
No
Action
071
SH: Semantics of length=0
16/12: Tabular description of semantics for all lengthKinds, including nil handling and defaults.
072
TK: Byte Order Mark and Unicode signature
16/12: Investigate whether the spec's position on UTF-16/32 BOM is implementable

Current Actions:
No
Action
045
20/05 AP: Speculative Parsing
27/05: Psuedo code has been circulated. Review for next call
03/06: Comments received and will be incorporated
09/06: Progress but not discussed
17/06: Discussed briefly
24/06: No Progress
01/07: No Progress
15/07: No progress. MB not happy with the way the algorithm is documented, need to find a better way.
29/07: No Progress
05/08: No Progress. Will document behaviour as a set of rules.
12/08: No Progress
...
16/09: no progress
30/09: AP distributed proposal and others commented. Brief discussion AP to incorporate update and reissue
07/10: Updated proposal was discussed.Comments will be incorporated into the next version.
14/10: Alan to update proposal to include array scenario where minOccurs > 0
21/10: Updated proposal reviewed
28/10: Updated proposal reviewed see minutes
04/11: Discussed semantics of disciminators on arrays. MB to produce examples
11/11: Absorbing action 033 into 045.  Maybe decorated discrminator kinds are needed after all. MB and SF to continue with examples.  
18/11: Went through WTX implementation of example. SF to gather more documentation about WTX discriminator rules.
25/11: Further discussion. Will get more WTX documentation. Need to confirm that no changes need to Resolving Uncertainty doc.
04/11: Further discussion about arrays.
09/12: Reviewed proposed discriminator semantic.
16/12: Reviewed discriminator examples and WTX semantic.
049
20/05 AP Built-in specification description and schemas
03/06: not discussed
24/06: No Progress
24/06: No Progress (hope to get these from test cases)
15/07: No progress. Once available, the examples in the spec should use the dfdl:defineFormat annotations they provide.
...
14/10: no progress
21/10: Discussed the real need for this being in the specification. It seemed that the main value is it define a schema location for downloading 'known' defaults from the web.
28/10: no progress
04/11: no progress
11/11: no update
18/11: no update
25/11: Agreed to try to produce for CSV and fixed formats
04/12: no update
09/12: no update
16/12: no update
056
MB Resolve lengthUnits=bits including fillbytes
12/08: No Progress
...
28/10: no progress
04/11: MB to look at lengthUnits = bits
11/11: no update
18/11: no update
25/11: no update
04/12: no update. ALan will set up a separate call to progress this action.
09/12: no update. ALan will set up a separate call to progress this action.
16/12: MB, SH and AP had  a separate call. MB to distribute proposal
064
MB/SH Request WG presentation at OGF 28
25/11: Session requested
04/12: no update
09/12: no update
16/12: SH has changed request to a general session rather than WG chartered session in the hope of attracting more people.
066
Investigate format for defining test cases
25/11:IBM to see if it is possible to publish its test case format.
04/12: no update
09/12: no update
16/12: reminded dent to project manager
068
Should the roots of messages be designated.?
09/12: Yes. New dfdl:documentRoot property
Closed
16/12: reopened and decided to drop property subject to agreement from SKK and SF

Closed actions
No
Action

Work items:
No
Item target version status
005
Improvements on property descriptions not started
011
How speculative parsing works (combining choice and variable-occurence - currently these are separate) (from action 045) awaiting completion of actions 045  
012
Reordering the properties discussion: move representation earlier, improve flow of topics not started
036
Update dfdl schema with change properties ongoing
038
Improve length section including bit handling some improvement in 036
042
Mapping of the DFDL infoset to XDM none not required for V1 specification
069
ICU fractional seconds
070
Write DFDL primer
071
Write test cases.
072
it is a processing error if the number of occurrences in the data does not match the value of the expression or prefix
073
Rename dfdl:separatorPolicy="required" to "always".
074
- Last 'postFix' separator is not optional
- Terminators are mandatory.
- dfdl:documentFinalTerminatorCanBeMissing
- dfdl:documentFinalSeparatorCanBeMissing  (Action (70))
075
Remove occursCountKind="useAvailableSpace".
076
 dfdl:documentRoot,  will be defined that can only be on global elements.
The DFDL spec does not have to define the format of parameters to the DFDL processor but will indicate that it must be possible to adresss any element.
Agreed that ANY element within the schema cane be the starting point for parsing or unparsing.
077
 'delimited' means the item is delimited by the item’s terminator (if specified) or an enclosing construct’s separator or end of the enclosing construct designated by its known length or its terminator.  
The definition of EndOfParent also needs improving.
078
document UPA checks



Alan Powell

MP 211, IBM UK Labs, Hursley,  Winchester, SO21 2JN, England
Notes Id: Alan Powell/UK/IBM     email: alan_powell@uk.ibm.com  
Tel: +44 (0)1962 815073                  Fax: +44 (0)1962 816898



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU