- dfdl-wg - lists.ogf.org

Re: [DFDL-WG] dfdl-wg Digest, Vol 43, Issue 2
by Tim Kimber 08 Mar '10

08 Mar '10

I have a bunch of questions/issues relating to dfdl:choiceKind. I'm not asking for changes in v0.40, but I expect there will be changes required. The issues that I want to raise are: a) The description of the property in v0.39 contains several typos and inaccuracies. - 'implicit' is being used where 'fixedLength' was intended. - nothing is said about the units in which the length is calculated. - there's no need to discuss how the choice is resolved when discussing the 'variableLength' enum - we should use standard phraseology when indicating whether a property can be computed from a DFDL expression. b) Property name should be 'choiceLengthKind' to accurately reflect its meaning c) There is a need for a related property 'choiceLengthUnits' Consider the recursive algorithm for calculating the length of each branch. It needs to know whether it is calculating a length in bytes or characters. If the length is in bytes, then the length cannot be calculated for variable-width encodings. If the length is in characters, then the length cannot always be calculated reliably if there are raw byte values in the markup. d) The rules for calculating the max length of the choice are not provided. They are complex, and not at all obvious. Consider these issues: The length of a branch cannot be calculated if - there are any optional elements or variable-length arrays anywhere in the branch - any field in the branch has dfdl:alignment > "1" ( at least, I can't work out what the rules would be. The alignment of the parent element would need to be factored in ) - any element or group in the branch specifies its initiator, terminator or separator as a DFDL expression - any element or group in the branch specifies its length as a DFDL expression if choiceLengthUnits='characters' then the length cannot be calculated if - any element or group in the branch specifies a DFDL string literal containing DFDL mnemonics %NL; %WSP*; or %WSP+; - any element or group in the branch uses a DFDL string literal that contains sequence of raw byte values with length different from the fixed character width if choiceLengthUnits='characters' then the length cannot be calculated if - any element in the branch specifies a variable-width encoding, or specifies its encoding as a DFDL expression. There are probably other rules which need to be applied, but the above should illustrate the point. Calculating the length is only possible under some *very* restrictive conditions. e) I think the property may not be required As far as I am aware, this property was introduced to provide support for COBOL REDEFINES, and to allow MRM message sets to be migrated to DFDL. If true, the problem gets a lot simpler: - COBOL does not use initiators/terminators. - The COBOL compiler contains code that calculates the length of the structure ( it must, because COBOL has a rule that a REDEFINES cannot be longer than the record that it is redefining ). Presumably, it takes alignment into account in some way, and handles issues relating to character width as well. - COBOL does not allow an anonymous REDEFINES. If imported, A REDEFINES will always produce a complex element whose content is a fixed-length choice. Note : This means that the same will be true of any MRM message set created by message broker's COBOLimporter. If those assumptions are correct, then in all cases the same effect could be achieved by putting the precalculated length of the REDEFINES onto the parent element. I think this merits serious consideration. The cost of implementing choiceKind='fixedLength' is quite high because of the complexity of the rules, and the fact that groups, as well as complex elements, can have a fixed length. But it's not really an implementation issue, it's a complexity issue. DFDL should not contain a propery with such complex implementation requirements unless there's a strong case for it - otherwise potential implementers are going to be put off. The existing COBOL importer probably does not set the precalculated length of a REDEFINES on the parent element. That would be required if we wanted to remove the property - so we would have to discuss that with the group that provides the importer technology. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

Final call for comments on DFDL V1 draft 039
by Alan Powell 03 Mar '10

03 Mar '10

This is the final call for comments on DFDL V1 draft 039 to be sent to me by 5 March 2010. http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.dfdl-wg/d… Draft 040, which will incorporate comments on draft 039, will be submitted to OGF for approval next week. Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Minutes for OGF DFDL Working Group Call, March 03-2010
by Alan Powell 03 Mar '10

03 Mar '10

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, March 03-2010 Attendees Suman Kalia (IBM) Steve Hanson (IBM) Alan Powell (IBM) Steve Marting (Progeny) Peter Lambros (IBM) Stephanie Fetzer (IBM) Tim Kimber(IBM) Apologies Mike Beckerle (Oco) 1. 16.2 scannablility with lengthKind pattern: In summary, you can use a data pattern on any element (complex, simple text, simple binary) as long as the bytes are legal in the stated encoding, which where binary data is involved in practice means an 8-bit ASCII encoding. Binary data can be handled using some of the conveniences of text by way of treating it as text with encoding="iso-8859-1". In this case literal text, such as length patterns, is interpreted as in the iso-8859-1 character encoding, and the correspondence of byte values in the data to a string in the DFDL infoset is one to one. That is, byte with value N, produces an infoset character with character code N. The WG agreed to the solution of recommending the use of "iso-8859-1" for binary fields. The write-up should note that he main use case is for a complex element that contains binary fields. 2. Current Actions: Updated below 3 Steve H issues with draft 039 1) Name of property dfdl:textNumberRepresentation is not consistent with dfdl:binaryNumberRep, dfdl:binaryFloatRep, etc. Agreed 2) The dfdl:numberPattern etc properties that have been moved from the defunct dfdl:textNumberFormat annotation to dfdl:element etc should be called dfdl:textNumberPattern etc. Otherwise users will think they apply to binary numbers too. Agreed 3) In section 14.3 on sequences, there are several sub-sections that talk about parsing according to different ways of specifying length (ie, lengthKind). But dfdl:sequence no longer carries dfdl:lengthKind so I think these sub-sections are not in the right place. I think they should be in section 12, under the correct 12.3.x lengthKind sub-section. MB has moved these sections 4) Section 19 on built-in specifications. Given that we don't have any for public comment phase we should reword this section. Agreed 4 Tim's (major) issues with draft 039 12.2 Delimiters: Text Markup - The term 'Delimiters' is not accurate. Most readers will not think of an initiator as a 'delimiter'. - It's not 'Text' markup any more - especially since v0.39 has allowed lengthKind="delimited" for elements with binary representation. Title should be 'Markup' and explanation can then deal with what it really is, rather than justifying the innaccurate title :-) Will be changed to Delimiters - Add delimiters to glossary (DFDLlimits delimiters to terminators and separator when dfdl:lengthKind=delimited) Syntax for specifying markup: It's not clear from this description that each item in the space-separated list is a DFDL string literal. Section 6.3.1 will be revised to separate string literals and dfdl expressions etc. If possible will add the list of property types (listof DFDL string literals, etc) and update property tables initiator ( and all other space-separated properties ) It is not clear whether the order of the space-separated properties matters. Must the parser test them in the order in which they are specified? ( Q: What if %ES; is the first in the list? ) The parser must look for the longest first. Unparsing will output the first in the list. The %ES entity should be not be allowed for initiator, separator and terminator. Should probably be limited to use in nilValues but will check whether it is useful in other properties.. occursStopValue should a value in the value space of the base type of the array. terminator: is it OK if the final terminator is missing within the scope of a known-length parent? Seems like a reasonable extension of the rule ( in all other scenarios, the end of a known-length parent acts like the end of the data stream for items with its scope ). documentFinalTerminatorCanBeMissing: Let's try to avoid creating another property for the postfix separator scenario. I think this property provides a way of modelling the data naturally. We can recommend use of infix-with-a-terminator rather than 'postfix' if the final terminator can be missing. outputNewLine Should we validate that the 'characterOrCharacters' are all newline characters from the set described by the %NL; mnemonic? Otherwise the DFDL serializer will output data which cannot be parsed by the DFDL parser. Agreed dfdl:lengthKind endOfParent 'endOfParent' has almost the same meaning as 'delimited' so should have the same semantics. · the item?s terminator (if specified) · an enclosing construct?s separator or terminator · the end of an enclosing construct designated by its known length · the end of the data stream The effect would be the the element could be ended by the nearest known length parent not just the immediate parent. Also the immediate parent could have lengthKind 'implicit' Not discussed but emails indicate Agreed choiceKind 'Fixed' When lengthKind='implicit' all alternative branches of the choice are padded to the fixed length of the largest one so that overall the entire choice construct is fixed length There must be a restriction that the length of at least one choice must be statically defined. Not discussed but emails indicate Agreed 5 DFDL v1 Specification completion. Draft 40 will be published early next week. Will be sent to OGF next week prior to OGF 28 Meeting closed, 14:10 Next call Wednesday 10 March January 2010 13:00 UK Next action: 084 Actions raised at this meeting No Action Current Actions: No Action 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 24/03: No progress 03/03: Discussions have been taking place on the subset of tests that will be provided. 080 AP:Clarify semantics of fn:poisition and fn:count 17/02: no progress 24/03: No progress 03/03: no progress. There are other functions which return a duration which need investgating Closed actions No Action 049 20/05 AP Built-in specification description and schemas 03/06: not discussed 24/06: No Progress 24/06: No Progress (hope to get these from test cases) 15/07: No progress. Once available, the examples in the spec should use the dfdl:defineFormat annotations they provide. ... 14/10: no progress 21/10: Discussed the real need for this being in the specification. It seemed that the main value is it define a schema location for downloading 'known' defaults from the web. 28/10: no progress 04/11: no progress 11/11: no update 18/11: no update 25/11: Agreed to try to produce for CSV and fixed formats 04/12: no update 09/12: no update 16/12: no update 23/12: no update 06/01: no progress. If there is no resource to complete this action it can be deferred 13/01:no progress 20/01: no progress 27/01: no progress 29/01: No progress. The predefined formats do not need to be available when the spec is published. Suman said that he had been mapping COBOL structures to DFDL and it didn't look as though the way text numbers are define is very usable. He will document for next call 03/02: No progress 10/02: No progress 17/03: No progress 24/03: No progress 03/03: The wording of this section will be changed to say that 'implementations may choose to provide predefined formats. The DFDL WG intends to supply a limited set' Closed 079 MB:Encoding for binary fields when lenghtkind is pattern 17/02: Discussed but no conclusion 24/03: Mike has found an encoding that matches the first 255 codepoints of iso 10646. Will document its use for binary fields. 03/03: Wording in minutes agreed. Closed 083 MB:To correct syntax diagram for FinalUnused and suggest wording for the Sequence section 03/03: Mike has supplied updates to Sytax, length and Sequence sections. Closed Work items: No Item target version status 005 Improvements on property descriptions not started 012 Reordering the properties discussion: move representation earlier, improve flow of topics not started 036 Update dfdl schema with change properties ongoing 042 Mapping of the DFDL infoset to XDM none not required for V1 specification 070 Write DFDL primer 071 Write test cases. 083 Implement RFC2116 084 MB:Encoding for binary fields when lenghtkind is pattern 085 MB:To correct syntax diagram for FinalUnused and suggest wording for the Sequence section 086 dfdl:textNumberRepresentation to dfdl:textNumberRep 087 dfdl:NumberPattern etc to dfdl:textNumberPattern (not for calendarformat but check individual properties) 088 Update built-in specifications section. 089 12.2 Will be changed to Delimiters - Add delimiters to glossary (DFDLlimits delimiters to terminators and separator when dfdl:lengthKind=delimited) 090 Section 6.3.1 will be revised to separate string literals and dfdl expressions etc. If possible will add the list of property types (listof DFDL string literals, etc) and update property tables 091 The parser must look for the longest first. Unparsing will output the first in the list. The %ES entity should be not be allowed for initiator, separator and terminator. Should probably be limited to use in nilValues but will check whether it is useful in other properties.. 092 occursStopValue should a value in the value space of the base type of the array. 093 outputNewLine MUST be one of newline characters from the set described by the %NL; mnemonic 094 dfdl:lengthKind endOfParent 'endOfParent' has almost the same meaning as 'delimited' so should have the same semantics. · the item?s terminator (if specified) · an enclosing construct?s separator or terminator · the end of an enclosing construct designated by its known length · the end of the data stream The effect would be the the element could be ended by the nearest known length parent not just the immediate parent. Also the immediate parent could have lengthKind 'implicit' 095 ChoiceKinf fixed:There must be a restriction that the length of at least one choice must be statically defined. 096 The WG agreed to the solution of recommending the use of "iso-8859-1" for binary fields. The write-up should note that he main use case is for a complex element that contains binary fields. Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] dfdl-wg Digest, Vol 43, Issue 2
by Tim Kimber 03 Mar '10

03 Mar '10

I need to attend the sprint planning meeting from 2pm GMT, so I may not be around when these items are discussed. Comments added below: 12.2 Delimiters: Text Markup - The term 'Delimiters' is not accurate. Most readers will not think of an initiator as a 'delimiter'. - It's not 'Text' markup any more - especially since v0.39 has allowed lengthKind="delimited" for elements with binary representation. Title should be 'Markup' and explanation can then deal with what it really is, rather than justifying the innaccurate title :-) I dislike the use of the term "markup" for something not written by people, and most data formats of the DFDL kind are written by computers, so nothing is getting "marked up" by anyone. Initiators certainly are delimiters in the situations where they are not tags. I.e., initiator="[" terminator="]". Only tags will not be thought of as delimiters. Even then I think it is a stretch to say that nobody will think of an introductory tag as a delimiter. These definitions found online: de·lim·it·er (d?-l?m'?-t?r) n. Computer Science A character or sequence of characters marking the beginning or end of a unit of data. Computing Dictionary delimiter character A character or string used to separate, or mark the start and end of, items of data in, e.g., a database, source code, or text file. See also: record. (2001-03-16) These definitions are consistent with our usage of the term. I suggest no change in our terminology here. <TK> Point taken re: the term 'delimiter'. I still have reservations about calling it 'Text Markup' in the title, though. I think the intro paragraph should explain the common usage ( intiators, separators, terminators for text formats ) and the exceptional usage ( handling delimited binary data and other non-text markup ) </TK> Syntax for specifying markup: It's not clear from this description that each item in the space-separated list is a DFDL string literal. These have always bugged me. Any better solution is welcome. XML/XSD does tend to make space separated the standard way to specify more than one thing. <TK> In a future revision of the spec we need a list of property value types which can then be used consistently in the tables which describe properties. - Enumeration - DFDL string literal - List of DFDL string literals - DFDL expression - DFDL regular expression - Boolean - Non-negative integer - any more? In some cases it will be necessary to place restrictions on the type of content allowed in the string literal ( disallow raw byte values / raw byte values must represent a character / etc ) </TK initiator ( and all other space-separated properties ) It is not clear whether the order of the space-separated properties matters. Must the parser test them in the order in which they are specified? ( Q: What if %ES; is the first in the list? ) I think the order should not matter, and it should test them longest first. <TK> Good idea. I have another related suggestion below. </TK> terminator: is it OK if the final terminator is missing within the scope of a known-length parent? Seems like a reasonable extension of the rule ( in all other scenarios, the end of a known-length parent acts like the end of the data stream for items with its scope ). I believe this should be true. "Final" is relative in my mind. <TK> Good - it's much easier to implement if end of known length parent is always equivalent to end of data stream, from the point of view of enclosed elements. But see next point... </TK> documentFinalTerminatorCanBeMissing: Let's try to avoid creating another property for the postfix separator scenario. I think this property provides a way of modelling the data naturally. We can recommend use of infix-with-a-terminator rather than 'postfix' if the final terminator can be missing. Copasetic. <TK> Had to look up 'copasetic'. I'm amazed that my Mum never came out with that one - she's a walking dictionary. This property has caused problems with naming and interpretation all along the line. Last time we discussed it, I don't think we considered this option ( we did talk about something like it ): - If %ES; is included in the list of values for separator or terminator then a) The parser ignores it while performing ordinary scanning ( otherwise it would always cause a zero-length string to be scanned ). b) The parser accepts 'end of data stream' as a match for the %ES; mnemonic. That makes this property ( and the equivalent one for separators ) redundant. c) Other usages of %ES; remain unchanged. </TK> outputNewLine Should we validate that the 'characterOrCharacters' are all newline characters from the set described by the %NL; mnemonic? Otherwise the DFDL serializer will output data which cannot be parsed by the DFDL parser. Nice catch. dfdl:lengthKind endOfParent 'endOfParent' has almost the same meaning as 'delimited' so should have the same semantics. · the item?s terminator (if specified) · an enclosing construct?s separator or terminator · the end of an enclosing construct designated by its known length · the end of the data stream The effect would be the the element could be ended by the nearest known length parent not just the immediate parent. Also the immediate parent could have lengthKind 'implicit' Agreed. choiceKind 'Fixed' When lengthKind='implicit' all alternative branches of the choice are padded to the fixed length of the largest one so that overall the entire choice construct is fixed length There must be a restriction that the length of at least one choice must be statically defined. Also good catch. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Agenda for OGF DFDL WG call 3 March 2010- 13:00 UK (8:00 ET)
by Alan Powell 02 Mar '10

02 Mar '10

1. 16.2 scannablility with lengthKind pattern: 2. Current Actions: 3 Steve H issues with draft 039 4 Tim's (major) issues with draft 039 5 Status of specification (for OGF28) 1. 16.2 scannablility with lengthKind pattern: In summary, you can use a data pattern on any element (complex, simple text, simple binary) as long as the bytes are legal in the stated encoding, which where binary data is involved in practice means an 8-bit ASCII encoding. Binary data can be handled using some of the conveniences of text by way of treating it as text with encoding="iso-8859-1". In this case literal text, such as length patterns, is interpreted as in the iso-8859-1 character encoding, and the correspondence of byte values in the data to a string in the DFDL infoset is one to one. That is, byte with value N, produces an infoset character with character code N. 2. Current Actions: No Action 049 20/05 AP Built-in specification description and schemas 03/06: not discussed 24/06: No Progress 24/06: No Progress (hope to get these from test cases) 15/07: No progress. Once available, the examples in the spec should use the dfdl:defineFormat annotations they provide. ... 14/10: no progress 21/10: Discussed the real need for this being in the specification. It seemed that the main value is it define a schema location for downloading 'known' defaults from the web. 28/10: no progress 04/11: no progress 11/11: no update 18/11: no update 25/11: Agreed to try to produce for CSV and fixed formats 04/12: no update 09/12: no update 16/12: no update 23/12: no update 06/01: no progress. If there is no resource to complete this action it can be deferred 13/01:no progress 20/01: no progress 27/01: no progress 29/01: No progress. The predefined formats do not need to be available when the spec is published. Suman said that he had been mapping COBOL structures to DFDL and it didn't look as though the way text numbers are define is very usable. He will document for next call 03/02: No progress 10/02: No progress 17/03: No progress 24/03: No progress 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 24/03: No progress 079 MB:Encoding for binary fields when lenghtkind is pattern 17/02: Discussed but no conclusion 24/03: Mike has found an encoding that matches the first 255 codepoints of iso 10646. Will document its use for binary fields. 080 AP:Clarify semantics of fn:poisition and fn:count 17/02: no progress 24/03: No progress 083 MB:To correct syntax diagram for FinalUnused and suggest wording for the Sequence section 3 Steve H issues with draft 039 1) Name of property dfdl:textNumberRepresentation is not consistent with dfdl:binaryNumberRep, dfdl:binaryFloatRep, etc. 2) The dfdl:numberPattern etc properties that have been moved from the defunct dfdl:textNumberFormat annotation to dfdl:element etc should be called dfdl:textNumberPattern etc. Otherwise users will think they apply to binary numbers too. 3) In section 14.3 on sequences, there are several sub-sections that talk about parsing according to different ways of specifying length (ie, lengthKind). But dfdl:sequence no longer carries dfdl:lengthKind so I think these sub-sections are not in the right place. I think they should be in section 12, under the correct 12.3.x lengthKind sub-section. 4) Section 19 on built-in specifications. Given that we don't have any for public comment phase we should reword this section. 4 Tim's (major) issues with draft 039 12.2 Delimiters: Text Markup - The term 'Delimiters' is not accurate. Most readers will not think of an initiator as a 'delimiter'. - It's not 'Text' markup any more - especially since v0.39 has allowed lengthKind="delimited" for elements with binary representation. Title should be 'Markup' and explanation can then deal with what it really is, rather than justifying the innaccurate title :-) Syntax for specifying markup: It's not clear from this description that each item in the space-separated list is a DFDL string literal. initiator ( and all other space-separated properties ) It is not clear whether the order of the space-separated properties matters. Must the parser test them in the order in which they are specified? ( Q: What if %ES; is the first in the list? ) terminator: is it OK if the final terminator is missing within the scope of a known-length parent? Seems like a reasonable extension of the rule ( in all other scenarios, the end of a known-length parent acts like the end of the data stream for items with its scope ). documentFinalTerminatorCanBeMissing: Let's try to avoid creating another property for the postfix separator scenario. I think this property provides a way of modelling the data naturally. We can recommend use of infix-with-a-terminator rather than 'postfix' if the final terminator can be missing. outputNewLine Should we validate that the 'characterOrCharacters' are all newline characters from the set described by the %NL; mnemonic? Otherwise the DFDL serializer will output data which cannot be parsed by the DFDL parser. dfdl:lengthKind endOfParent 'endOfParent' has almost the same meaning as 'delimited' so should have the same semantics. · the item?s terminator (if specified) · an enclosing construct?s separator or terminator · the end of an enclosing construct designated by its known length · the end of the data stream The effect would be the the element could be ended by the nearest known length parent not just the immediate parent. Also the immediate parent could have lengthKind 'implicit' choiceKind 'Fixed' When lengthKind='implicit' all alternative branches of the choice are padded to the fixed length of the largest one so that overall the entire choice construct is fixed length There must be a restriction that the length of at least one choice must be statically defined. Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

DFDL V1 Draft 039 is available
by Alan Powell 02 Mar '10

02 Mar '10

All DFDL V1 draft 038 is available at https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.dfdl-wg/… There have again been many changes listed below including many editorial updates to make the specification more readable. The DFDL expression language and DFDL regular expression language sections have been moved towards the end as they are for more advanced users. All highlighted changes are for the changes made in this draft only and do not include changes made in previous drafts. Unfortunately I have had significant problems with MS Word crashes while creating this draft which have resulted the formatting going haywire in some places. I have corrected the major problems but many minor ones still exist. In particular all the cross references have lost the number of the section they reference. This will be fixed in the next draft. Please send comments to me by March 3rd Latest entry at the top please Version Author/ Contributor History Date(yyyy-mm-dd) 039 Alan Powell Added numberPattern section. Added defaulting of complex types Changed missingValueInitiatorPolicy to 'required' and 'prohibited' Changed separatorPolicy to 'required' 'supressed' 'supressedAtEnd' Added defaulting choices. Many editorial changes Added lengthKind delimited for 'binary' 'packed/bcd' Clarified empty sequqnce wording Reworded infoset round tripping. Moved DFDL Expression and regular expression section to near back os spec. Added treatment of unrepresentable characaters to infoset section. Clatified relative path expressions on global declarations. Representation assumed to be text for string and binary for hexBinary Change alignment to be 1 based. docmentFinalTerninatorCanBeMissing and docmentFinalSeparatorCanBeMissing change to dfdl:format only Added UPA checks Added Ambiguity checks Removed defineTextNumberFormat, TextNumberFormat and textNumberFormatRef and moved properties in line Revised 'Resolving points of uncertainty' and discriminators Clarified Nan and infinity values 2010-02-24 Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

3 2

Minutes for OGF DFDL Working Group Call, February 24-2010
by Alan Powell 24 Feb '10

24 Feb '10

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, February 24-2010 Attendees Mike Beckerle (Oco) Suman Kalia (IBM) Steve Hanson (IBM) Alan Powell (IBM) Steve Marting (Progeny) Peter Lambros (IBM) Stephanie Fetzer (IBM) Apologies Tim Kimber(IBM) 1. Remaining 037 review issues A: 16.2 scannablility with lengthKind pattern: Confirm that this is what we agreed In summary, you can use a data pattern on any element (complex, simple text, simple binary) as long as the bytes are legal in the stated encoding, which where binary data is involved in practice means an 8-bit ASCII encoding. Mike B I found an official reference which has no "greyed out" codepoints. All 256 values are "mapped". The following ftp table (see URL below) officially defines the mapping for 8859-1 to unicode/iso10646. The table includes all 256 codepoints - some are specified as just <control> i.e., have no specific meaning, but their 8859 codepoint maps one-to-one and onto a unicode/10646 codepoint with the same value. Note that this property holds for 8859-1. It does not hold for 8859-2 to 8859-16, as these have character codes substituted into them that map to other places in the iso10646 codepoint space. Here's the correspondence table: ftp://ftp.unicode.org/Public/MAPPINGS/ISO8859/8859-1.TXT If we reference this mapping table in the references of the DFDL spec, then I believe we can say that using encoding="iso-8859-1", you can treat binary data as textual, use patterns, etc., and the relationship to/from the infoset always insures preservation of the values of the bytes (parsing), and creation of bytes whose values exactly match the string codepoints (unparsing). This language can be added to the section on lengthKind="pattern" and binary data: Binary data can be handled using some of the conveniences of text by way of treating it as text with encoding="iso-8859-1". In this case literal text, such as length patterns, is interpreted as in the iso-8859-1 character encoding, and the correspondence of byte values in the data to a string in the DFDL infoset is one to one. That is, byte with value N, produces an infoset character with character code N. [reference to above FTP site]. B: Glossary Variable-Occurrence Item - Optional elements have a variable number of occurrences (0 or 1) and arrays also can have a variable number of occurrences (when minOccurs < maxOccurs). So when we say an item with a variable number of occurrences, this can mean either an optional element, or an array where minOccurs < maxOccurs. In either array or optional elements, we have the additional constraint that the DFDL representation properties do not preclude a variable number of occurrences. When dfdl:occursCountKind='explicit' and dfdl:occursCount has a literal constant as its value, or an expression that statically evaluates to a constant, then the DFDL properties are specifying exactly the number of occurrences for all instances and so are said to preclude a variable number of occurrences. If dfdl:occursCount has a formula as its expressed value, then the DFDL properties do not preclude a variable number of occurrences. MikeB Comment: This idea that you can have minOccurs < maxOccurs, but dfdl:occurs is equal to a constant and dfdl:occursKind="explicit" is causing us a bunch of grief in these definitions. Can we be conservative and just say it is a schema definition error if minOccurs < maxOccurs but the length is static, i.e., an explicit constant-valued expression? WG decided that the wording can remain as currently written C: DFDL Schema Component Model Only changes needed are: remove wildcards. Add following to describe shading: The shaded boxes have a direct corresponding element syntax and therefore appear in DFDL schema D: Sequence Groups Mike B: To correct syntax diagram for FinalUnused and suggest wording for the Sequence section E: Check other comments in document. Please look at the remaining comments in draft 039 and suggest solutions 2. Go through Actions Updated below 3 DFDL v1 Specification completion. Draft 039 will be publish today. WG review and Comments by 3 March Draft 40 with updates for OGF submission - available 5 March Meeting closed, 14:00 Next call Wednesday 3 March January 2010 13:00 UK Next action: 084 Actions raised at this meeting No Action 083 MB:To correct syntax diagram for FinalUnused and suggest wording for the Sequence section Current Actions: No Action 049 20/05 AP Built-in specification description and schemas 03/06: not discussed 24/06: No Progress 24/06: No Progress (hope to get these from test cases) 15/07: No progress. Once available, the examples in the spec should use the dfdl:defineFormat annotations they provide. ... 14/10: no progress 21/10: Discussed the real need for this being in the specification. It seemed that the main value is it define a schema location for downloading 'known' defaults from the web. 28/10: no progress 04/11: no progress 11/11: no update 18/11: no update 25/11: Agreed to try to produce for CSV and fixed formats 04/12: no update 09/12: no update 16/12: no update 23/12: no update 06/01: no progress. If there is no resource to complete this action it can be deferred 13/01:no progress 20/01: no progress 27/01: no progress 29/01: No progress. The predefined formats do not need to be available when the spec is published. Suman said that he had been mapping COBOL structures to DFDL and it didn't look as though the way text numbers are define is very usable. He will document for next call 03/02: No progress 10/02: No progress 17/03: No progress 24/03: No progress 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 24/03: No progress 079 MB:Encoding for binary fields when lenghtkind is pattern 17/02: Discussed but no conclusion 24/03: Mike has found an encoding that matches the first 255 codepoints of iso 10646. Will document its use for binary fields. 080 AP:Clarify semantics of fn:poisition and fn:count 17/02: no progress 24/03: No progress 081 AP: Inf and Nan The description is the way ICU behaves but need clarification. It isn't clear how inf and Nan are represented in the infoset. Need to investigate if XML allows these values 17/02: XML allows Nan and inf for float and double Dfdl will do the same. Requires more investigation of ICU. 24/03: Alan send clarification that Inf and Nan are limited to float and Double. Closed. 083 MB:To correct syntax diagram for FinalUnused and suggest wording for the Sequence section Closed actions No Action 081 AP: Inf and Nan The description is the way ICU behaves but need clarification. It isn't clear how inf and Nan are represented in the infoset. Need to investigate if XML allows these values 17/02: XML allows Nan and inf for float and double Dfdl will do the same. Requires more investigation of ICU. 24/03: Alan send clarification that Inf and Nan are limited to float and Double. Closed. Work items: No Item target version status 005 Improvements on property descriptions not started 012 Reordering the properties discussion: move representation earlier, improve flow of topics not started 036 Update dfdl schema with change properties ongoing 042 Mapping of the DFDL infoset to XDM none not required for V1 specification 069 ICU fractional seconds 039 070 Write DFDL primer 071 Write test cases. 072 it is a processing error if the number of occurrences in the data does not match the value of the expression or prefix 039 073 Rename dfdl:separatorPolicy="required" to "always". 039 Defferred untilaction 071 agreed 078 document UPA checks 039 079 Semantics of length=0, nil handling and defaults. (A071) 039 080 Tlog: Allow LengthKind delimited for packed/bcd (A074) 039 081 Update empty sequence section (A075) 039 082 semantics of minOccurs= 0 on choice branches (A076) 039 083 Implement RFC2116 084 Length|Kind pattern scanability rules 039 085 Invalid character substitution 039 086 infoset round tripi: Rephrase sentence 'It is possible to define a schema so that when infoset unparsed and the datastream reparsed, the same infoset will be produced' 039 087 Clarify use of relative paths in global components. 039 088 'DFDL expression' 039 089 Ageed that dfdl:represetnation 'text' is implied for strings and dfdl:represetnation 'binary' is implied for hexbinary 039 091 textStringPadCharacter textNumberPadCharacter must be a 1 byte character if the char set encoding is variable width? 039 092 finalDocumentTerminatorCanBeMissing and finalDocumentSeparatorCanBeMissing allowed only in 'default' format 039 093 remove textNumberFormat and textCalendarFormat. 039 094 Alignment should be 1 based 039 095 AP: Inf and Nan 039 Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] 8-bit-ascii for dealing with binary data in text-like manner - problematic
by Mike Beckerle 24 Feb '10

24 Feb '10

I think we've got a fix for this. I found an official reference which has no "greyed out" codepoints. All 256 values are "mapped". The following ftp table (see URL below) officially defines the mapping for 8859-1 to unicode/iso10646. The table includes all 256 codepoints - some are specified as just <control> i.e., have no specific meaning, but their 8859 codepoint maps one-to-one and onto a unicode/10646 codepoint with the same value. Note that this property holds for 8859-1. It does not hold for 8859-2 to 8859-16, as these have character codes substituted into them that map to other places in the iso10646 codepoint space. Here's the correspondence table: ftp://ftp.unicode.org/Public/MAPPINGS/ISO8859/8859-1.TXT If we reference this mapping table in the references of the DFDL spec, then I believe we can say that using encoding="iso-8859-1", you can treat binary data as textual, use patterns, etc., and the relationship to/from the infoset always insures preservation of the values of the bytes (parsing), and creation of bytes whose values exactly match the string codepoints (unparsing). This language can be added to the section on lengthKind="pattern" and binary data: Binary data can be handled using some of the conveniences of text by way of treating it as text with encoding="iso-8859-1". In this case literal text, such as length patterns, is interpreted as in the iso-8859-1 character encoding, and the correspondence of byte values in the data to a string in the DFDL infoset is one to one. That is, byte with value N, produces an infoset character with character code N. [reference to above FTP site]. On Thu, Feb 11, 2010 at 5:32 AM, Steve Hanson <smh(a)uk.ibm.com> wrote: > > Mike > > In the wikipedia entry for ISO 10646 it says "The system deliberately > leaves many code points not assigned to characters, even in the BMP. It does > this to allow for future expansion or to minimize conflicts with other > encoding forms." If those code points are below 256 then we have the same > problem as 8859? I can't find an actual map of the 10646 code points - you > have to buy it from ISO. > > Regards > > Steve Hanson > Programming Model Architect, WebSphere Message Broker, > OGF DFDL WG Co-Chair, > Hursley, UK, > Internet: smh(a)uk.ibm.com, > Phone (+44)/(0) 1962-815848 > > > From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: dfdl-wg(a)ogf.org Date: 11/02/2010 > 00:38 Subject: [DFDL-WG] 8-bit-ascii for dealing with binary data in > text-like manner - problematic Sent by: dfdl-wg-bounces(a)ogf.org > ------------------------------ > > > > > Every "8-bit-ascii" encoding I can find has holes in the code page. That > is, values that don't have a corresponding character codepoint assigned. > > Example: iso-8859-X are a bunch of 8-bit ascii-based encodings that are > popular. > > If you lookup iso-8859-1 it has this language: > > Code values 00–1F, 7F–9F are not assigned to characters by ISO/IEC 8859-1. > > The lower range 20 to 7E (the G0 subset) maps exactly to the same coded G0 > subset of the ISO 646 US variant (commonly known as *ASCII*<http://en.wikipedia.org/wiki/ASCII>), > ... > > They're saying 7-bit ascii is included, and some other codes are there, but > they don't assign a codepoint generally. > > So, to me suggesting use of any particular code page for this purpose is > somewhat ambiguous. E.g., what does &#x01 mean in a string if the encoding > is iso-8859-1? There appears to be a set of translation tables that assign > this to unicode in standard ways that one can find on the web. But the > codepoint doesn't have an assigned meaning in iso-8859-X standards. > > Two possible clarifications: > 1) for all ascii-based character sets, we say that bytes 0x00 to 0xFF all > map to exactly those codepoints in ISO 10646 for the infoset, and vice > versa. > > 2) define dfdl:encoding="bytes" as a special character set name which has > the above property. > > Personally, I prefer 2. It is simpler to explain what is going on, and when > people are depending on bytes it will be clearer that they are. > > ...mike > > -- > dfdl-wg mailing list > dfdl-wg(a)ogf.org > http://www.ogf.org/mailman/listinfo/dfdl-wg > > > > > ------------------------------ > > * > * > > *Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > * > > > > > > >

1 0

Agenda for OGF DFDL WG call 24 February 2010- 13:00 UK (8:00 ET)
by Alan Powell 24 Feb '10

24 Feb '10

1. Remaining 037 review issues A: 16.2 scannablility with lengthKind pattern: Confirm that this is what we agreed In summary, you can use a data pattern on any element (complex, simple text, simple binary) as long as the bytes are legal in the stated encoding, which where binary data is involved in practice means an 8-bit ASCII encoding. By 8-bit ASCII we really mean an encoding where all the codepoints from 0-255 map to the equivalent value. Subsequent investigation indicates that 'all' 8-bit ASCII encodings have gaps so there isn't a valid character. Mike has suggested 1) for all ascii-based character sets, we say that bytes 0x00 to 0xFF all map to exactly those codepoints in ISO 10646 for the infoset, and vice versa. 2) define dfdl:encoding="bytes" as a special character set name which has the above property. Briefly discussed but no conclusion. B: Glossary Variable-Occurrence Item - Optional elements have a variable number of occurrences (0 or 1) and arrays also can have a variable number of occurrences (when minOccurs < maxOccurs). So when we say an item with a variable number of occurrences, this can mean either an optional element, or an array where minOccurs < maxOccurs. In either array or optional elements, we have the additional constraint that the DFDL representation properties do not preclude a variable number of occurrences. When dfdl:occursCountKind='explicit' and dfdl:occursCount has a literal constant as its value, or an expression that statically evaluates to a constant, then the DFDL properties are specifying exactly the number of occurrences for all instances and so are said to preclude a variable number of occurrences. If dfdl:occursCount has a formula as its expressed value, then the DFDL properties do not preclude a variable number of occurrences. MikeB Comment: This idea that you can have minOccurs < maxOccurs, but dfdl:occurs is equal to a constant and dfdl:occursKind="explicit" is causing us a bunch of grief in these definitions. Can we be conservative and just say it is a schema definition error if minOccurs < maxOccurs but the length is static, i.e., an explicit constant-valued expression? C: DFDL Schema Component Model What needs to be changed in the UML diagram? D: Sequence Groups Mike B: TBD: rewrite these property descriptions in terms of the grammar for sequences Specifically, this is where the FinalUnused Region must be described. E: Check other comments in document. 2. Go through Actions Current Actions: No Action 049 20/05 AP Built-in specification description and schemas 03/06: not discussed 24/06: No Progress 24/06: No Progress (hope to get these from test cases) 15/07: No progress. Once available, the examples in the spec should use the dfdl:defineFormat annotations they provide. ... 14/10: no progress 21/10: Discussed the real need for this being in the specification. It seemed that the main value is it define a schema location for downloading 'known' defaults from the web. 28/10: no progress 04/11: no progress 11/11: no update 18/11: no update 25/11: Agreed to try to produce for CSV and fixed formats 04/12: no update 09/12: no update 16/12: no update 23/12: no update 06/01: no progress. If there is no resource to complete this action it can be deferred 13/01:no progress 20/01: no progress 27/01: no progress 29/01: No progress. The predefined formats do not need to be available when the spec is published. Suman said that he had been mapping COBOL structures to DFDL and it didn't look as though the way text numbers are define is very usable. He will document for next call 03/02: No progress 10/02: No progress 17/03: No progress 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 079 MB:Encoding for binary fields when lenghtkind is pattern 17/02: Discussed but no conclusion 080 AP:Clarify semantics of fn:poisition and fn:count 17/02: no progress 081 AP: Inf and Nan The description is the way ICU behaves but need clarification. It isn't clear how inf and Nan are represented in the infoset. Need to investigate if XML allows these values 17/02: XML allows Nan and inf for float and double Dfdl will do the same. Requires more investigation of ICU. 3 DFDL v1 Specification completion. Draft 039 will be publish today. WG review and Comments by 3 March Draft 40 with updates for OGF submission - available 5 March Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Minutes for OGF DFDL Working Group Call, February 17-2010
by Alan Powell 18 Feb '10

18 Feb '10

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, February 17-2010 Attendees Steve Hanson (IBM) Alan Powell (IBM) Steve Marting (Progeny) Peter Lambros (IBM) Stephanie Fetzer (IBM) Apologies Mike Beckerle (Oco) Suman Kalia (IBM) Tim Kimber(IBM) 1. Comments of latest discriminators doc v5. Steve questioned: It is a processing error if none of the choice branches are known to exist Steve: I know we agreed on this but I?m concerned that I am now unable to model the situation where I have a header, a footer, and in between either nothing or exactly 1 of a choice of n records It was agreed that the body could be an optional complex element with the choice as its content. Also confirmed dfdl:discriminator can be an annotation on · a local xs:element declaration · an xs:element reference · an xs:group reference (when the top level of a choice branch) · an xs:sequence (when the top level of a choice branch) · an xs:choice (when the top level of a choice branch) Action 045 will be closed. 2. Remaining 037 review issues 16.2 scannablility with lengthKind pattern: Confirm that this is what we agreed In summary, you can use a data pattern on any element (complex, simple text, simple binary) as long as the bytes are legal in the stated encoding, which where binary data is involved in practice means an 8-bit ASCII encoding. By 8-bit ASCII we really mean an encoding where all the codepoints from 0-255 map to the equivalent value. Subsequent investigation indicates that 'all' 8-bit ASCII encodings have gaps so there isn't a valid character. Mike has suggested 1) for all ascii-based character sets, we say that bytes 0x00 to 0xFF all map to exactly those codepoints in ISO 10646 for the infoset, and vice versa. 2) define dfdl:encoding="bytes" as a special character set name which has the above property. Briefly discussed but no conclusion. 3. Go through Actions Updated below 4. Biztalk comparison Short discussion which noted that DFDL could match Biztalk representation properties but was missing support for attributes and some simple types. Decided that we don't need to do anything in DFDL v1. 5 DFDL v1 Specification completion. It was agreed that the spec should be in the OGF review process by OGF 28 in March. Alan stated it needed 2 more drafts. Draft 39 with all the updates completed for a final WG review - available 24 Feb Comments by 3 March Draft 40 with updates for OGF submission - available 5 March Meeting closed, 14:00 Next call Wednesday 24 February January 2010 13:00 UK Next action: 083 Actions raised at this meeting No Action Current Actions: No Action 049 20/05 AP Built-in specification description and schemas 03/06: not discussed 24/06: No Progress 24/06: No Progress (hope to get these from test cases) 15/07: No progress. Once available, the examples in the spec should use the dfdl:defineFormat annotations they provide. ... 14/10: no progress 21/10: Discussed the real need for this being in the specification. It seemed that the main value is it define a schema location for downloading 'known' defaults from the web. 28/10: no progress 04/11: no progress 11/11: no update 18/11: no update 25/11: Agreed to try to produce for CSV and fixed formats 04/12: no update 09/12: no update 16/12: no update 23/12: no update 06/01: no progress. If there is no resource to complete this action it can be deferred 13/01:no progress 20/01: no progress 27/01: no progress 29/01: No progress. The predefined formats do not need to be available when the spec is published. Suman said that he had been mapping COBOL structures to DFDL and it didn't look as though the way text numbers are define is very usable. He will document for next call 03/02: No progress 10/02: No progress 17/03: No progress 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 079 MB:Encoding for binary fields when lenghtkind is pattern 17/02: Discussed but no conclusion 080 AP:Clarify semantics of fn:poisition and fn:count 17/02: no progress 081 AP: Inf and Nan The description is the way ICU behaves but need clarification. It isn't clear how inf and Nan are represented in the infoset. Need to investigate if XML allows these values 17/02: XML allows Nan and inf for float and double Dfdl will do the same. Requires more investigation of ICU. Closed actions No Action 045 20/05 AP: Speculative Parsing 27/05: Psuedo code has been circulated. Review for next call 03/06: Comments received and will be incorporated 09/06: Progress but not discussed 17/06: Discussed briefly 24/06: No Progress 01/07: No Progress 15/07: No progress. MB not happy with the way the algorithm is documented, need to find a better way. 29/07: No Progress 05/08: No Progress. Will document behaviour as a set of rules. 12/08: No Progress ... 16/09: no progress 30/09: AP distributed proposal and others commented. Brief discussion AP to incorporate update and reissue 07/10: Updated proposal was discussed.Comments will be incorporated into the next version. 14/10: Alan to update proposal to include array scenario where minOccurs > 0 21/10: Updated proposal reviewed 28/10: Updated proposal reviewed see minutes 04/11: Discussed semantics of disciminators on arrays. MB to produce examples 11/11: Absorbing action 033 into 045. Maybe decorated discrminator kinds are needed after all. MB and SF to continue with examples. 18/11: Went through WTX implementation of example. SF to gather more documentation about WTX discriminator rules. 25/11: Further discussion. Will get more WTX documentation. Need to confirm that no changes need to Resolving Uncertainty doc. 04/11: Further discussion about arrays. 09/12: Reviewed proposed discriminator semantic. 16/12: Reviewed discriminator examples and WTX semantic. 23/12: SF to provide better description of WTX behaviour and invite B Connolley to next call 06/01:B Connolly not available. SF to provide more complete description. 13/01: Stephaine took us through a description of WTX identifiers. Mike agreed to write up in DFDL terms. 20/01: Mike will write up 27/01: further discussion of discriminators 29/01: Alan had emailed both proposals but not enough time to discuss 02/02: Agreed to adopt 'component exists' semantics for discriminators 10/02: 'component exists' proposal updated. comments by next call. 17/02: reviewed needs minor updates. Closed 082 MB: Should alignment be 0 or 1 based 17/02: Mike has reworded the section to be 1 based and Agreed this was OK Closed Work items: No Item target version status 005 Improvements on property descriptions not started 012 Reordering the properties discussion: move representation earlier, improve flow of topics not started 036 Update dfdl schema with change properties ongoing 042 Mapping of the DFDL infoset to XDM none not required for V1 specification 069 ICU fractional seconds 039 070 Write DFDL primer 071 Write test cases. 072 it is a processing error if the number of occurrences in the data does not match the value of the expression or prefix 039 073 Rename dfdl:separatorPolicy="required" to "always". 039 Defferred untilaction 071 agreed 078 document UPA checks 039 079 Semantics of length=0, nil handling and defaults. (A071) 039 080 Tlog: Allow LengthKind delimited for packed/bcd (A074) 039 081 Update empty sequence section (A075) 039 082 semantics of minOccurs= 0 on choice branches (A076) 039 083 Implement RFC2116 084 Length|Kind pattern scanability rules 085 Invalid character substitution 086 infoset round tripi: Rephrase sentence 'It is possible to define a schema so that when infoset unparsed and the datastream reparsed, the same infoset will be produced' 087 Clarify use of relative paths in global components. 088 'DFDL expression' 089 Ageed that dfdl:represetnation 'text' is implied for strings and dfdl:represetnation 'binary' is implied for hexbinary 091 textStringPadCharacter textNumberPadCharacter must be a 1 byte character if the char set encoding is variable width? 092 finalDocumentTerminatorCanBeMissing and finalDocumentSeparatorCanBeMissing allowed only in 'default' format 093 remove textNumberFormat and textCalendarFormat. 094 Alignment should be 1 based Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0