- dfdl-wg - lists.ogf.org

WG call this week
by Alan Powell 22 Mar '10

22 Mar '10

I cannot make the WG call this week and as we don't have anything urgent to discuss I propose that we cancel it. Let me know if you would prefer it to go ahead. Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Minutes for OGF DFDL Working Group Call, March 17-2010
by Alan Powell 18 Mar '10

18 Mar '10

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, March 17-2010 Attendees Suman Kalia (IBM) Steve Hanson (IBM) Alan Powell (IBM) Steve Marting (Progeny) Stephanie Fetzer (IBM) Mike Beckerle (Oco) Apologies Tim Kimber(IBM) Steve H reported on the DFDL presentation at OGF 28 Overall OGF registered attendees thought to be around 250. Number at DFDL session: 7. Apart from Erwin (Data AD), DFDL was new to everyone. Notes from session (from minutes taken by Erwin Laure based on questions asked): Validation on input and output? Means conforming to the schema defined (e.g. integer between 0 and 100). dfdl assert could be used for asserting correctness of the data. Further complex validation would be via an external step (eg, Schematron invocation). Using DFDL to model data structures generated by OO code, particularly use of inheritance? Could be a use case for allowing complex type inheritance in DFDL in the future. Data often comes with units, infoset would want to reflect this? Could be achieved using expressions, dfdl:inputValueCalc and dfdl:hidden to apply a scaling factor based on units. What's the efficiency of DFDL? Particularly, can tests be turned off for fast read/write? The spec defines the behavior of the parser but not how to implement it. For instance, validation is not mandatory. Open source reference implementation would be good to have. (Lots of nodding). Scientific floating point data compresses badly. Knowing data structure can allow a more intelligent compression. DFDL is not intending to do transformations but that could be done on top. Encryption/compression could also be a use case for multi-layers, or for additional functions in the expression language. Need to reach out to DAIS-WG and DR-WG for (public) comments and to see whether DFDL will be actively used by those groups. Open source implementation would obviously help here. Comment on the spec: Should be a proposed recommendation (GFD-P-R) not informational (GFD-I). Draft 040 has been submitted to OGF and will be reviewed by the technical committee on March 30th. It is expected to go out for public comment in early April. There needs to be some publicity to ensure that enough comments are made. Mike suggested a press release from his company with IBM comment. Action raised May be able to get IBM representatives on other standards bodies such as ACCORD. OMG, to interest their members, Should contact all the contributors to the WG to ask them to review. Mike will update the information he sent in response to an RFI from OMG 1 Process for dealing with internal issues during Public comment phase. The public comments process will be used to make updates 2. Nils and Defaults during unparsing The table 17 in section 13.16.2 was corrected in draft 40 but here are still some ambiguities, for example when nil is the default. Logical Value nilValueInitiatorPolicy Has default value specified missingValueInitiatorPolicy initiator region contains content region contains Nil (implies nillable) prohibited don't care don't care empty representation of nil based on nilKind, nilValue, etc. required initiator string "" (empty string) Note that this implies that the element type is xs:string don't care prohibited empty empty string required initiator string a non-nil non-empty-string value don't care don't care initiator string The representation of the logical value Not supplied don?t care Yes (non-empty sting) don't care Initiator string The representation of the default value. Yes (empty string) prohibited empty empty required initiator string empty Alan will go update the table. 3 dfdl:choiceKind The main issues are: a) The calculation of the length of the longest branch is not obvious. b) The length units to use - the dfdl:lengthUnits property does not exist on a choice c) The name could be better Proposal is therefore to retain the property but to: i) State the conditions that must apply to use this property, and enforce them in the validator => schema definition error otherwise ii) Decouple the choice from its parent by calculating the length of each branch based solely on the properties of the branches components, irrespective of any parent dfdl:lengthKind Alan document the problem. Name change agreed. 4 DFDL time functions fn:timezone-from-dateTime Returns the timezone from an xs:dateTime value. fn:timezone-from-date Returns the timezone from an xs:date value. fn:timezone-from-time Returns the timezone from an xs:time value. Function Meaning fn:adjust-dateTime-to-timezone Adjusts an xs:dateTime value to a specific timezone, or to no timezone at all. fn:adjust-date-to-timezone Adjusts an xs:date value to a specific timezone, or to no timezone at all. fn:adjust-time-to-timezone Adjusts an xs:time value to a specific timezone, or to no timezone at all. All return an xs:duration. Do we need these functions? As there is no known use case for these functions they will be dropped. Meeting closed, 14:10 Next call Wednesday 17 March January 2010 13:00 UK (9:00 ET) Call will be for one hour only NOTE: East coast is 4 hours behind the UK for the next two weeks Next action: 087 Actions raised at this meeting No Action 085 ALL: publicize Public comments phase to ensure a good review.. 086 AP: Nils and Defaults during unparsing - update table Current Actions: No Action 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 24/03: No progress 03/03: Discussions have been taking place on the subset of tests that will be provided. 10/03: work is progressing 17/03: work is progressin 084 Check behaviour of dfdl:inputValueCalc and outputValueCalc. 085 ALL: publicize Public comments phase to ensure a good review.. 086 AP: Nils and Defaults during unparsing - update table Closed actions No Action Work items: No Item target version status 005 Improvements on property descriptions not started 012 Reordering the properties discussion: move representation earlier, improve flow of topics not started 036 Update dfdl schema with change properties ongoing 042 Mapping of the DFDL infoset to XDM none not required for V1 specification 070 Write DFDL primer 071 Write test cases. 083 Implement RFC2116 097 Remove functions that returns duration Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: Minutes from DFDL session at OGF 28
by Steve Hanson 17 Mar '10

17 Mar '10

Overall OGF registered attendees thought to be around 250. Number at DFDL session: 7. Apart from Erwin (Data AD), DFDL was new to everyone. Notes from session (from minutes taken by Erwin Laure based on questions asked): Validation on input and output? Means conforming to the schema defined (e.g. integer between 0 and 100). dfdl assert could be used for asserting correctness of the data. Further complex validation would be via an external step (eg, Schematron invocation). Using DFDL to model data structures generated by OO code, particularly use of inheritance? Could be a use case for allowing complex type inheritance in DFDL in the future. Data often comes with units, infoset would want to reflect this? Could be achieved using expressions, dfdl:inputValueCalc and dfdl:hidden to apply a scaling factor based on units. What's the efficiency of DFDL? Particularly, can tests be turned off for fast read/write? The spec defines the behavior of the parser but not how to implement it. For instance, validation is not mandatory. Open source reference implementation would be good to have. (Lots of nodding). Scientific floating point data compresses badly. Knowing data structure can allow a more intelligent compression. DFDL is not intending to do transformations but that could be done on top. Encryption/compression could also be a use case for multi-layers, or for additional functions in the expression language. Need to reach out to DAIS-WG and DR-WG for (public) comments and to see whether DFDL will be actively used by those groups. Open source implementation would obviously help here. Comment on the spec: Should be a proposed recommendation (GFD-P-R) not informational (GFD-I). Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Agenda for OGF DFDL WG call 3 March 2010- 13:00 UK (9:00 ET)
by Alan Powell 16 Mar '10

16 Mar '10

NOTE: The east coast is 4 hours behind the UK for the next two weeks so start time is 9:00 am ET 1 Process for dealing with internal issues during Public comment phase. 2. Nils and Defaults during unparsing The table 17 in section 13.16.2 was corrected in draft 40 but here are still some ambiguities, for example when nil is the default. Logical Value nilValueInitiatorPolicy Has default value specified missingValueInitiatorPolicy initiator region contains content region contains Nil (implies nillable) prohibited don't care don't care empty representation of nil based on nilKind, nilValue, etc. required initiator string "" (empty string) Note that this implies that the element type is xs:string don't care prohibited empty empty string required initiator string a non-nil non-empty-string value don't care don't care initiator string The representation of the logical value Not supplied don?t care Yes (non-empty sting) don't care Initiator string The representation of the default value. Yes (empty string) prohibited empty empty required initiator string empty 3 dfdl:choiceKind The main issues are: a) The calculation of the length of the longest branch is not obvious. b) The length units to use - the dfdl:lengthUnits property does not exist on a choice c) The name could be better Proposal is therefore to retain the property but to: i) State the conditions that must apply to use this property, and enforce them in the validator => schema definition error otherwise ii) Decouple the choice from its parent by calculating the length of each branch based solely on the properties of the branches components, irrespective of any parent dfdl:lengthKind 4 DFDL time functions fn:timezone-from-dateTime Returns the timezone from an xs:dateTime value. fn:timezone-from-date Returns the timezone from an xs:date value. fn:timezone-from-time Returns the timezone from an xs:time value. Function Meaning fn:adjust-dateTime-to-timezone Adjusts an xs:dateTime value to a specific timezone, or to no timezone at all. fn:adjust-date-to-timezone Adjusts an xs:date value to a specific timezone, or to no timezone at all. fn:adjust-time-to-timezone Adjusts an xs:time value to a specific timezone, or to no timezone at all. All return an xs:duration. Do we need these functions? 5 Current Actions: No Action 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 24/03: No progress 03/03: Discussions have been taking place on the subset of tests that will be provided. 10/03: work is progressing 084 Check behaviour of dfdl:inputValueCalc and outputValueCalc. Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Minutes for OGF DFDL Working Group Call, March 10-2010
by Alan Powell 16 Mar '10

16 Mar '10

Open Grid Forum: Data Format Description Language Working Group OGF DFDL Working Group Call, March 10-2010 Attendees Suman Kalia (IBM) Steve Hanson (IBM) Alan Powell (IBM) Steve Marting (Progeny) Stephanie Fetzer (IBM) Tim Kimber(IBM) Apologies Mike Beckerle (Oco) 1. Spec issues documentFinalTerminatorCanBeMissing: Let's try to avoid creating another property for the postfix separator scenario. I think this property provides a way of modelling the data naturally. We can recommend use of infix-with-a-terminator rather than 'postfix' if the final terminator can be missing. WG deferred any changes the property at this stage dfdl:lengthKind endOfParent 'endOfParent' has almost the same meaning as 'delimited' so should have the same semantics. · the item?s terminator (if specified) · an enclosing construct?s separator or terminator · the end of an enclosing construct designated by its known length · the end of the data stream The effect would be the the element could be ended by the nearest known length parent not just the immediate parent. Also the immediate parent could have lengthKind 'implicit' Agreed with the definition and that dfdl:lengthKind 'endOfParent should apply to simpleTypes only. dfdl:lengthUnits 'characaters' LengthUnits 'characters' is currently for simple representation 'text' only. Should it be allowed for complex? Agreed dfdl:lengthUnits should be allowed on complex elements. The characters are in the encoding of the complex element. The endocings of the children must be the same as the parent.. Selectors Should they be moved to an appendix. Agreed should moved DFDL Expression Syntax Value comparison operators are 'eq' 'lt' 'gt' etc not '=' '<' '>' because these were defined poorly in XPATH v1 and replaced in XPATH V2. Unfortunately most examples us '=' etc The examples need changing and '=' '<' will not be accepted Syntax of expressions in string literals. Add A DFDL string literal which begins with { and does not end with } does not need to have its leading { escaped. The syntax rules for expressions are · Must start with a '{' in the first position and end with '}' in the last position. · '{' in any other position if treated as a literal · '}' in any position other than the last position is treated as a literal. · '{{' as the first characters are treated as the literal '{' and not a DFDL expression. IgnoreCase Should ignoreCase apply to numberInfinityRep and numberNaNRep Agreed Escapes. Should escapeCharacter, escapeEscapeCharacater etc allow DFDL expressions. Agreed OutputNewLine should allow an expression Agreed Clarify dfdl: InputValueCalc and outputValueCalc It is a schema definition error if both inputvalueCalc and outputValueCalc are specified. InputValueCalc is examined on unparsing so needs to be added to unparsing property precedence. Action to review inputValueCalc and putputValueCalc 2. Current Actions: Updated below 3 Status of specification (for OGF28) Draft 40 will be published by tomorrow and submitted to OGF on friday. Meeting closed, 14:10 Next call Wednesday 10 March January 2010 13:00 UK (9:00 ET) NOTE: East coast is 4 hours behind the UK for the next two weeks Next action: 085 Actions raised at this meeting No Action 084 Check behaviour of dfdl:inputValueCalc and outputValueCalc. Current Actions: No Action 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 24/03: No progress 03/03: Discussions have been taking place on the subset of tests that will be provided. 10/03: work is progressing 084 Check behaviour of dfdl:inputValueCalc and outputValueCalc. Closed actions No Action 080 AP:Clarify semantics of fn:poisition and fn:count 17/02: no progress 24/03: No progress 03/03: no progress. There are other functions which return a duration which need investgating 10/03: changed to dfdl:position and dfdl:count. closed Work items: No Item target version status 005 Improvements on property descriptions not started 012 Reordering the properties discussion: move representation earlier, improve flow of topics not started 036 Update dfdl schema with change properties ongoing 042 Mapping of the DFDL infoset to XDM none not required for V1 specification 070 Write DFDL primer 071 Write test cases. 083 Implement RFC2116 Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

OGF 28: DFDL 1.0 Public Comment session
by Steve Hanson 15 Mar '10

15 Mar '10

Finalised details of this Wednesday's session at OGF 28 in Munich are here http://www.ogf.org/gf/event_schedule/index.php?id=1933, including slides. Regards Steve Hanson Programming Model Architect, WebSphere Message Broker, Co-Chair, OGF DFDL WG Hursley, UK, Internet: smh(a)uk.ibm.com, Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Data Format Description Language specification submitted to OGF for review.
by Alan Powell 12 Mar '10

12 Mar '10

All The Data Format Description Language V1 specification has been submitted to the OGF committee for review. There were a couple of minor editorial changed and a not-so-minor update to the grammar production table from the draft sent out yesterday. Updated versions can be found at Draft 040: https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.dfdl-wg/… Submitted version: https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.dfdl-wg/… Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL V1 Draft 040 and OGF submission copies available
by Alan Powell 11 Mar '10

11 Mar '10

DFDL v1 draft 040 is available at https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.dfdl-wg/… The version to be submitted to OGF is at https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.dfdl-wg/… This is the same as draft 040 but with the revision history removed and all the changes accepted If there are no serious comments, this draft will be submitted to OGF tomorrow afternoon. As far as I am aware all the issues and most of the editorial comments have been incorporated. Revision History Latest entry at the top please Version Author/ Contributor History Date(yyyy-mm-dd) 040 Alan Powell Mike Beckerle Steve Hanson Tim Kimber dfdl:textNumberRepresentation changed to textNumberRep numberXXX from textNumberFormat changed to textNumberXXX Fixed up grammar (double tables), dropped grammar for simple types (wasn't used anywhere) replacing it with a single grammar terminal 'simpleRepresentation' which is used for all the simple types. Modified properties to refer to grammar terminals (some - not done comprehensively). Moved sections from sequences that have to do with specified length, and pattern length that rightfully aren't just about sequences. These were moved up to the general section about Framing and length. Change all boolean properties to enum 'yes''no' Added textNumberGroupSeparator Corrected Nils and Deafults on Unparsing table Clarified the valid types for properties Moved selectors to appendix Added dfdl:inputValueCalc to unparsing property precedence. Changed all examples to use 'eq' 'gt' rather than '=' '>' Allow dfdl:lengthUnits 'characters' on complex elements Allow escapeCharacter and escapeEscapeCharacater to be an expression. outputNewLine can be an expression fixed examples added copyright statements for ICU and XPATH change fn:count and fn:position to dfdl:count and dfdl:position Regards Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Agenda for OGF DFDL WG call 10 March 2010- 13:00 UK (8:00 ET)
by Alan Powell 10 Mar '10

10 Mar '10

1. Spec issues documentFinalTerminatorCanBeMissing: Let's try to avoid creating another property for the postfix separator scenario. I think this property provides a way of modelling the data naturally. We can recommend use of infix-with-a-terminator rather than 'postfix' if the final terminator can be missing. We didn't come to a conclusion on the alst WG call. dfdl:lengthKind endOfParent 'endOfParent' has almost the same meaning as 'delimited' so should have the same semantics. · the item?s terminator (if specified) · an enclosing construct?s separator or terminator · the end of an enclosing construct designated by its known length · the end of the data stream The effect would be the the element could be ended by the nearest known length parent not just the immediate parent. Also the immediate parent could have lengthKind 'implicit' dfdl:lengthKind endOfParent LengthUnits 'characters' is currently for simple representation 'text' only. Should it be allowed for complex? Selectors Should they be moved to an appendix. DFDL Expression Syntax Value comparison operators are 'eq' 'lt' 'gt' etc not '=' '<' '>' because these were defined poorly in XPATH v1 and replaced in XPATH V2. Unfortunately most examples us '=' etc Syntax of expressions in string literals. Add A DFDL string literal which begins with { and does not end with } does not need to have its leading { escaped. IgnoreCase Should ignoreCase apply to numberInfinityRep and numberNaNRep Escapes. Should escapeCharacter, escapeEscapeCharacater etc allow DFDL expressions. 2. Current Actions: No Action 066 Investigate format for defining test cases 25/11:IBM to see if it is possible to publish its test case format. 04/12: no update 09/12: no update 16/12: reminded dent to project manager 23/12: SH will send another reminder. 06/01: Another reminder will be sent 13/01: no update 20/01: no update 27/01: no progress 29/01: no progress 03/02: IBM is still investigating 10/02: IBM is still investigating 17/02: IBM is willing in principle to publish the test case format and some of the test cases. May need some time to build a 'compliance suite' 24/03: No progress 03/03: Discussions have been taking place on the subset of tests that will be provided. 080 AP:Clarify semantics of fn:poisition and fn:count 17/02: no progress 24/03: No progress 03/03: no progress. There are other functions which return a duration which need investgating 3 Status of specification (for OGF28) Alan Powell Development - MQSeries, Message Broker, ESB IBM Software Group, Application and Integration Middleware Software ------------------------------------------------------------------------------------------------------------------------------------------- IBM MP211, Hursley Park Hursley, SO21 2JN United Kingdom Phone: +44-1962-815073 e-mail: alan_powell(a)uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] dfdl-wg Digest, Vol 43, Issue 2
by Tim Kimber 08 Mar '10

08 Mar '10

I have a bunch of questions/issues relating to dfdl:choiceKind. I'm not asking for changes in v0.40, but I expect there will be changes required. The issues that I want to raise are: a) The description of the property in v0.39 contains several typos and inaccuracies. - 'implicit' is being used where 'fixedLength' was intended. - nothing is said about the units in which the length is calculated. - there's no need to discuss how the choice is resolved when discussing the 'variableLength' enum - we should use standard phraseology when indicating whether a property can be computed from a DFDL expression. b) Property name should be 'choiceLengthKind' to accurately reflect its meaning c) There is a need for a related property 'choiceLengthUnits' Consider the recursive algorithm for calculating the length of each branch. It needs to know whether it is calculating a length in bytes or characters. If the length is in bytes, then the length cannot be calculated for variable-width encodings. If the length is in characters, then the length cannot always be calculated reliably if there are raw byte values in the markup. d) The rules for calculating the max length of the choice are not provided. They are complex, and not at all obvious. Consider these issues: The length of a branch cannot be calculated if - there are any optional elements or variable-length arrays anywhere in the branch - any field in the branch has dfdl:alignment > "1" ( at least, I can't work out what the rules would be. The alignment of the parent element would need to be factored in ) - any element or group in the branch specifies its initiator, terminator or separator as a DFDL expression - any element or group in the branch specifies its length as a DFDL expression if choiceLengthUnits='characters' then the length cannot be calculated if - any element or group in the branch specifies a DFDL string literal containing DFDL mnemonics %NL; %WSP*; or %WSP+; - any element or group in the branch uses a DFDL string literal that contains sequence of raw byte values with length different from the fixed character width if choiceLengthUnits='characters' then the length cannot be calculated if - any element in the branch specifies a variable-width encoding, or specifies its encoding as a DFDL expression. There are probably other rules which need to be applied, but the above should illustrate the point. Calculating the length is only possible under some *very* restrictive conditions. e) I think the property may not be required As far as I am aware, this property was introduced to provide support for COBOL REDEFINES, and to allow MRM message sets to be migrated to DFDL. If true, the problem gets a lot simpler: - COBOL does not use initiators/terminators. - The COBOL compiler contains code that calculates the length of the structure ( it must, because COBOL has a rule that a REDEFINES cannot be longer than the record that it is redefining ). Presumably, it takes alignment into account in some way, and handles issues relating to character width as well. - COBOL does not allow an anonymous REDEFINES. If imported, A REDEFINES will always produce a complex element whose content is a fixed-length choice. Note : This means that the same will be true of any MRM message set created by message broker's COBOLimporter. If those assumptions are correct, then in all cases the same effect could be achieved by putting the precalculated length of the REDEFINES onto the parent element. I think this merits serious consideration. The cost of implementing choiceKind='fixedLength' is quite high because of the complexity of the rules, and the fact that groups, as well as complex elements, can have a fixed length. But it's not really an implementation issue, it's a complexity issue. DFDL should not contain a propery with such complex implementation requirements unless there's a strong case for it - otherwise potential implementers are going to be put off. The existing COBOL importer probably does not set the precalculated length of a REDEFINES on the parent element. That would be required if we wanted to remove the property - so we would have to discuss that with the group that provides the importer technology. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1