November 2007 - dfdl-wg - lists.ogf.org

Re: [DFDL-WG] Grammar as adapted to infoset
by Steve Hanson 21 Nov '07

21 Nov '07

Allowing dfdl:lengthKind on choices has other benefits. It means that elements, sequences and choices are consistent in how they are capable of being extracted (literal length, XPath, model determined, prefixed, null terminated). I don't think unresolvableWhenParsing implies fixed length storage. I could have an unresolvable choice being two variable length strings, terminated by the terminator of the choice. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle/Worcester/IBM@IBMUS 20/11/2007 21:23 To Steve Hanson/UK/IBM@IBMGB cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Rather than overloading "implicit" length to mean this, I think we need an enum-valued property for choices. Today there is this choiceUnresolvableWhenParsing boolean. I think this shoudl become choiceType="unresolvableWhenParsing". Then we can have an enum value for the case where all arms of the choice are assumed to be padded to max length of the alternative. So I think we have 3 enum values: variableLength fixedLength unresolvableWhenParsing variableLength means the storage taken depends on the alternative selected, Fixed length means shorter alternatives are padded to the length of the longest. unresolvable means what it means now, which is that you can't tell which variant it is (which also implies the storage required is fixed length). ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/19/2007 07:13 AM To Mike Beckerle/Worcester/IBM@IBMUS cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Or allow xs:choice to carry dfdl:length & dfdl:lengthKind. Then lengthKind="implicit" could mean the length is the longest of the modelled alternatives. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle/Worcester/IBM@IBMUS 16/11/2007 19:29 To Steve Hanson/UK/IBM@IBMGB cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset There are of course alternatives. E.g., a hidden or non-hidden field which is a byte array or hexBinary to absorb the padding bytes at the ends of the shorter variants. Or one could put trailing skip bytes on the last element or group of each of the shorter variants. One can also encapsulate the whole choice in a sequence whose length is the fixed maximum length. This is my personal favorite, since it uses the FinalUnused that is already in the grammar. E.g., suppose the length of the longest variant is 258 bytes. <sequence dfdl:lengthKind="explicit" dfdl:length="258" dfdl:lengthUnits="bytes" dfdl:applies="hereOnly"> <choice> ..... </choice> </sequence> The only thing I don't like about this is I don't have a way to express the length other than to hard-code the constant 258, but since tooling would typically put this in based on Cobol descriptors or other information I don't mind so much. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/16/2007 11:24 AM To Mike Beckerle/Worcester/IBM cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Hi Mike One extra thought post-call, caused by thinking about the FinalUnused part of a sequence. Some fixed length choices have a rule that each branch of the choice must be of equal length. This results in unused bytes for branches shorter than the maximum. In a bitstream instance, a user will see these bytes. Perhaps the grammar should acknowledge their existence, even if there are no DFDL properties to express the bytes (like FinalUnused)? Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle <beckerle(a)us.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 10/11/2007 00:16 To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] Grammar as adapted to infoset I took a crack at this. I ripped it out of the main document for faster review/turnaround. I think this matches the latest infoset email diagram. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Grammar as adapted to infoset
by Mike Beckerle 20 Nov '07

20 Nov '07

Rather than overloading "implicit" length to mean this, I think we need an enum-valued property for choices. Today there is this choiceUnresolvableWhenParsing boolean. I think this shoudl become choiceType="unresolvableWhenParsing". Then we can have an enum value for the case where all arms of the choice are assumed to be padded to max length of the alternative. So I think we have 3 enum values: variableLength fixedLength unresolvableWhenParsing variableLength means the storage taken depends on the alternative selected, Fixed length means shorter alternatives are padded to the length of the longest. unresolvable means what it means now, which is that you can't tell which variant it is (which also implies the storage required is fixed length). ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/19/2007 07:13 AM To Mike Beckerle/Worcester/IBM@IBMUS cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Or allow xs:choice to carry dfdl:length & dfdl:lengthKind. Then lengthKind="implicit" could mean the length is the longest of the modelled alternatives. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle/Worcester/IBM@IBMUS 16/11/2007 19:29 To Steve Hanson/UK/IBM@IBMGB cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset There are of course alternatives. E.g., a hidden or non-hidden field which is a byte array or hexBinary to absorb the padding bytes at the ends of the shorter variants. Or one could put trailing skip bytes on the last element or group of each of the shorter variants. One can also encapsulate the whole choice in a sequence whose length is the fixed maximum length. This is my personal favorite, since it uses the FinalUnused that is already in the grammar. E.g., suppose the length of the longest variant is 258 bytes. <sequence dfdl:lengthKind="explicit" dfdl:length="258" dfdl:lengthUnits="bytes" dfdl:applies="hereOnly"> <choice> ..... </choice> </sequence> The only thing I don't like about this is I don't have a way to express the length other than to hard-code the constant 258, but since tooling would typically put this in based on Cobol descriptors or other information I don't mind so much. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/16/2007 11:24 AM To Mike Beckerle/Worcester/IBM cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Hi Mike One extra thought post-call, caused by thinking about the FinalUnused part of a sequence. Some fixed length choices have a rule that each branch of the choice must be of equal length. This results in unused bytes for branches shorter than the maximum. In a bitstream instance, a user will see these bytes. Perhaps the grammar should acknowledge their existence, even if there are no DFDL properties to express the bytes (like FinalUnused)? Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle <beckerle(a)us.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 10/11/2007 00:16 To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] Grammar as adapted to infoset I took a crack at this. I ripped it out of the main document for faster review/turnaround. I think this matches the latest infoset email diagram. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

1 0

DFDL infoset and hexBinary
by Mike Beckerle 20 Nov '07

20 Nov '07

What is the DFDL infoset value for a hexBinary type element? Our doc says an element of type hexBinary has a value member in the infoset whose type is a hexBinary, but what is a hexBinary anyway? Consider this 12 bytes of data, dumped here as hex: 003100320033004100420043 If that's a string in UTF-16BE encoding, it looks like "123ABC" It is 12 bytes long. Suppose we describe this using this DFDL: <element name="a" type="hexBinary" length='12' /> As I understand it, this is correct XSD in that the length of a hexBinary is always specified as the number of 'octets' of the binary data, not characters. Now, the contents of the basic XML infoset for an element matching this schema are these 24 characters: "003100320033004100420043". This is because the XML infoset has no idea what this string is because it is not schema aware. An XML document corresponding to this might look like: <! xml version='1.0' encoding="UTF-16BE" !> <a>003100320033004100420043</a> So, the question is...... What are the contents of the DFDL Infoset for a hexBinary like this? Same 24 characters? or 12 'octets' of data? This is essentially the answer to the question "what IS a hexBinary?" or maybe "what is a hexBinary in the PSVI other than just a string?" ...mikeb

2 2

DFDL hexBinary and base64Binary
by Mike Beckerle 20 Nov '07

20 Nov '07

I'm trying to wrap up the opaque/hexBinary/base64Binary topic. I need opinions on this discussion. Currently we have a property, dfdl:binaryType : Properties Specific to Binary Types (hexBinary, base64Binary) Property Name Description binaryType Enum This specifies the encoding method for the binary. Valid values are ‘unspecified’, ‘hexBinary’, ‘base64Binary’, ‘uuencoded’ Annotation: dfdl:element (simple type ‘binary’, ‘opaque’) This property speaks to what kinds of representations can we interpret and construct logical hexbinary values from? (similarly base64Binary) I believe the above is not clear, and causes issues with the xs:length facet of XSD. I propose the 4 tables below which describe the 4 cases: hexbinary - binary hexbinary - text base64binary - binary base64binary - text I have specified these so that the meaning of the xs:length facet is always interpreted exactly as in XSD. It always refers to the number of bytes of the unencoded binary data, and never to the number of characters in the encoded form. type representation lengthKind resulting length (in bytes) other xs:hexBinary binary implicit xs:length facet explicit dfdl:length Validation: xs:length facet must be equal to resulting length in bytes (TBD: similar range checks on xs:minLength, xs:maxLength) endOfData or delimited or nullTerminated variable type representation lengthKind resulting length (in characters) other xs:hexBinary text implicit 2 * xs:length facet explicit dfdl:length Validation: xs:length facet * 2 must be equal to resulting character length (after removing all non-hex characters) (TBD: similar range checks on xs:minLength, xs:maxLength) endOfData, delimited, nullTerminated Variable type representation dfdl:lengthKind resulting length (in bytes) other xs:base64Binary binary implicit xs:length facet explicit dfdl:length Validation: xs:length facet must be equal to resulting length in bytes (TBD: similar range checks on xs:minLength, xs:maxLength) endOfData or delimited or nullTerminated variable type representation lengthKind resulting length (in characters) other xs:base64Binary text implicit 8/6 * xs:length facet explicit dfdl:length Validation: xs:length facet * 8/6 must be equal to resulting character length (after removing all non-base64-encoding characters) (TBD: similar range checks on xs:minLength, xs:maxLength) endOfData, delimited, nullTerminated Variable Looking at the above, one way to simplify things quite a bit is to disallow the xs:length and xs:minLength and xs:maxLength facet on hexBinary and base64Binary types in DFDL schemas. Then the implicit lengthKind goes away, and the complex validation check for the xs:length facet goes away. I recommend this. Another simplification alternative is to disallow representation text altogether, but I am concerned that peopel with data that does contain hex or base64 data will naturally want to use these types to model it. I don't recommend this. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

4 7

Fw: DFDL array prefix
by Mike Beckerle 20 Nov '07

20 Nov '07

This is where Suman and I left it on the array prefix issue. The problem is where do you stop with creating special "occurs" versions. You can in principle have an "occurs" variant of almost every property. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 ----- Forwarded by Mike Beckerle/Worcester/IBM on 11/20/2007 10:02 AM ----- Suman Kalia/Toronto/IBM@IBMCA 11/16/2007 11:53 AM To Mike Beckerle/Worcester/IBM@IBMUS cc Subject Fw: DFDL array prefix Mike - The proposed grammar productions seem correct to me.. I see your view point and your concern for bifurcation of properties and I do agree that it introduces level of redundancy. I guess the choice boils down to having extra level of redundancy (dummy sequence) in the logical model or having redundancy in DFDL properties and each alternative can be justified and supported with good arguments. From consumability perspective having clean and less/no extra level of redundancy in the logical model is preferred because it would be at the user face all the time while the DFDL properties pertaining to occurrences will be rare and only for specific use cases. From pure DFDL spec perspective, having no redundancy in DFDL properties is desirable. May be we can go with the conservative approach as you mentioned ie not add the extra production in the grammar for Array for now but add an appendix in the document listing the alternative for overcoming redundancy in the logical model and how the revised grammar would look like. We can get more feedback when the document is reviewed by IBMers (architects, implementers, UCD) and folks in the standards community. Suman Kalia IBM Toronto Lab WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 11/16/2007 10:34 AM ----- Mike Beckerle/Worcester/IBM@IBMUS 11/16/2007 10:17 AM To Suman Kalia/Toronto/IBM@IBMCA cc Subject DFDL array prefix Here's the proposal I think: Array = ArrayPrefix ArrayContent ArrayContent = [ Element [ OccursSeparator Element ]* [ OccursSeparator StopValue] ] StopValue = SimpleElement I'd suggest the keyword occursLeadingSkipBytes is the property which controls the ArrayPrefix content. My argument against this is a slippery-slope argument. This starts us down the path of having a bifurcation of many properties into an occurs and non-occurs flavor. E.g., once you add the above, then by analogy why not have these additional keywords which apply to the array as a whole, not the elements of it: occursAlignment occursAlignmentUnits occursInitiator occursTerminator occursLength occursLengthKind occursLengthUnits All of this is avoided by simply requiring an extra sequence to provide these same format characteristics using the ordinary non-occurs keywords, and that's very preferable to me to this proliferation of occurs and non-occurs variants. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

1 0

DFDL - debate around array specific properties
by Steve Hanson 20 Nov '07

20 Nov '07

On Friday's call Mike & Suman were tasked with creating a proposal for the number of array specific properties that we expose in 1.0. Spec v028 lists a core set: occursKind occurs occursSeparator occursStopvalue occursStopValueKind The debate was whether array properties are needed for handling prefix/suffix region, the motivating example being leading skip count. Here's a format I came across recently, JSON, meaning JavaScriptObjectModel (http://json.org/) used in conjunction with REST style web services. For a model that logically looks like... String name; String[] address; String zip; ...an example JSON instance could look like... {"name":"Steve Hanson", "address":["1 Woodlands","Awbridge","Romsey"], "zip":"SO51 0GP" } ... and a DFDL model (ignoring white space) would look like. <xs:sequence dfdl:separator="," dfdl:initiator="{" dfdl:terminator="}" dfdl:lengthKind="implicit"> <xs:element name="name" type="xs:string" dfdl:initiator=""name":" dfdl:terminator="" dfdl:lengthKind="delimited"/> <xs:sequence dfdl:separator="" dfdl:initiator=""address":[" dfdl:terminator="]" dfdl:lengthKind="implicit"> <xs:element name="address" type="xs:string" dfdl:initiator="" dfdl:terminator="" dfdl:lengthKind="delimited" maxOccurs="unbounded" dfdl:occursKind="implicit" dfdl:occursSeparator=","/> </xs:sequence> <xs:element name="zip" type="xs:string" dfdl:initiator=""name":" dfdl:terminator=""/> </xs:sequence> Note the need to wrap the address element in a sequence to consume the array markup **. I could have used a complex element, but that would have added an extra infoset level and I wanted to conform as closely as possible to the logical view of the data above. To do this with explicit array properties the DFDL model would look like: <xs:element name="address" type="xs:string" dfdl:initiator="" dfdl:terminator="" dfdl:lengthKind="delimited" maxOccurs="unbounded" dfdl:occursKind="implicit" dfdl:occursSeparator="," dfdl:occursInitiator=""address":[" dfdl:occursTerminator="]" /> Note the need for dfdl:occursInitiator, dfdl:occursTerminator. Personally, I don't think the extra properties increase the useability by much. ** The one thing that feels a bit odd is having dfdl:lengthKind="implicit" on the wrapping xs:sequence. I feel like I want to use dfdl:lengthKind="delimited", because the sequence is followed by the separator of the outer sequence. And in this instance I could, as there are no 'final unused' bytes in the data. I'm still getting used to length on sequences, I guess. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 16 Nov 2007
by Ian W Parkinson 19 Nov '07

19 Nov '07

Open Grid Forum: Data Format Description Language Working Group Working Group Conference Call 14:00 GMT, 16 Nov 2007 Attendees Mike Beckerle (IBM) Geoff Judd (IBM) Steve Hanson (IBM) Suman Kalia (IBM) Ian Parkinson (IBM) Alan Powell (IBM) 1. Introduction Mike called the meeting to discuss the revised Data Grammar portion of the DFDL specification, which he had previously circulated. This was discussed on the regular, weekly, conference call on 14th November, in Mike's absence. Mike will not be able to attend the next scheduled conference call. 2. Review of DFDL Data Grammar The meeting discussed the following points: As noted in the previous meeting, both the ComplexContent and ComplexElement productions include a "Prefix" element. Mike stated that this redundancy does exist in DFDL, but it sometimes necessary, e.g. for embedded groups. Steve suggested that, if this did need to be changed, we would do so with restrictions rather than a change to the productions. Suman suggested that the present grammar specification does not satisfactorily cover the case where the start of an array needs to be aligned, with individual array elements automatically being aligned. Mike suggested that this can be dealt with, in a DFDL schema, by encapsulating the array in an element or sequence.This would require the author to build additional encapsulation around the array, which Suman felt would be unnatural, and suggested it would be more natural to introduce a property to explicitly deal with this situation. Mike observed that the XML specification treats arrays as simply adjacent elements, so we should minimize the number of properties which affect an entire array rather than each individual element. Mike and Suman will collaborate on a concrete proposal to address this. Mike expressed discomfort with the use of "binary" and "opaque" to describe raw data, as these words ,ay be ambiguous. "opaque" in particular may refer to unparsed data, or to anyType elements. Suggested alternatives included "octets", "bits", "raw" and "blob". The SequenceContent production currently contains a "finalUnused" element. Mike wondered whether this would best be moved to the Sequence production. Suman felt that there would then be issues with prefixes and suffixes. The meeting saw no good reason to change this. Precedence order needs to be defined. Steve and Geoff will attempt to define a static order. Regarding the query raised at the previous meeting concerning initial prefix Separators and final postfix Separators, Mike confirmed that these had been removed following comments in a previous draft of the document. They had originally been intended to support certain properties which could be now be defined using other features of the grammar. Meeting closed 15:00 GMT Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Grammar as adapted to infoset
by Steve Hanson 19 Nov '07

19 Nov '07

Or allow xs:choice to carry dfdl:length & dfdl:lengthKind. Then lengthKind="implicit" could mean the length is the longest of the modelled alternatives. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle/Worcester/IBM@IBMUS 16/11/2007 19:29 To Steve Hanson/UK/IBM@IBMGB cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset There are of course alternatives. E.g., a hidden or non-hidden field which is a byte array or hexBinary to absorb the padding bytes at the ends of the shorter variants. Or one could put trailing skip bytes on the last element or group of each of the shorter variants. One can also encapsulate the whole choice in a sequence whose length is the fixed maximum length. This is my personal favorite, since it uses the FinalUnused that is already in the grammar. E.g., suppose the length of the longest variant is 258 bytes. <sequence dfdl:lengthKind="explicit" dfdl:length="258" dfdl:lengthUnits="bytes" dfdl:applies="hereOnly"> <choice> ..... </choice> </sequence> The only thing I don't like about this is I don't have a way to express the length other than to hard-code the constant 258, but since tooling would typically put this in based on Cobol descriptors or other information I don't mind so much. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/16/2007 11:24 AM To Mike Beckerle/Worcester/IBM cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Hi Mike One extra thought post-call, caused by thinking about the FinalUnused part of a sequence. Some fixed length choices have a rule that each branch of the choice must be of equal length. This results in unused bytes for branches shorter than the maximum. In a bitstream instance, a user will see these bytes. Perhaps the grammar should acknowledge their existence, even if there are no DFDL properties to express the bytes (like FinalUnused)? Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle <beckerle(a)us.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 10/11/2007 00:16 To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] Grammar as adapted to infoset I took a crack at this. I ripped it out of the main document for faster review/turnaround. I think this matches the latest infoset email diagram. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Grammar as adapted to infoset
by Mike Beckerle 16 Nov '07

16 Nov '07

There are of course alternatives. E.g., a hidden or non-hidden field which is a byte array or hexBinary to absorb the padding bytes at the ends of the shorter variants. Or one could put trailing skip bytes on the last element or group of each of the shorter variants. One can also encapsulate the whole choice in a sequence whose length is the fixed maximum length. This is my personal favorite, since it uses the FinalUnused that is already in the grammar. E.g., suppose the length of the longest variant is 258 bytes. <sequence dfdl:lengthKind="explicit" dfdl:length="258" dfdl:lengthUnits="bytes" dfdl:applies="hereOnly"> <choice> ..... </choice> </sequence> The only thing I don't like about this is I don't have a way to express the length other than to hard-code the constant 258, but since tooling would typically put this in based on Cobol descriptors or other information I don't mind so much. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/16/2007 11:24 AM To Mike Beckerle/Worcester/IBM cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Hi Mike One extra thought post-call, caused by thinking about the FinalUnused part of a sequence. Some fixed length choices have a rule that each branch of the choice must be of equal length. This results in unused bytes for branches shorter than the maximum. In a bitstream instance, a user will see these bytes. Perhaps the grammar should acknowledge their existence, even if there are no DFDL properties to express the bytes (like FinalUnused)? Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle <beckerle(a)us.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 10/11/2007 00:16 To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] Grammar as adapted to infoset I took a crack at this. I ripped it out of the main document for faster review/turnaround. I think this matches the latest infoset email diagram. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

1 0

Grammar as adapted to infoset
by Mike Beckerle 16 Nov '07

16 Nov '07

I took a crack at this. I ripped it out of the main document for faster review/turnaround. I think this matches the latest infoset email diagram. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

2 1