- dfdl-wg - lists.ogf.org

OGF DFDL plans for next few spec versions
by Mike Beckerle 28 Nov '07

28 Nov '07

I'd like to rough out targets for content for the next several spec drafts: vNext - (I hope to get this all in this week) - binary/opaque types resolved - also anySimpleType - facets discussion integrated in - choiceType enum and length props on choice - prefix lengths - wildcards with dimension fix - xs:integer type added vX+1 - 2-level description (schema components) - nulls/default/optionals - regular expressions for lengths - expression language - valueCalc vX+2 - assertions/discriminators and choice - how speculative parsing works - reordering properties discussion (move representation earlier) vX+3 - escapeSchemes - string xml type - variables and selectors - improvments on property descriptions

1 0

OGF DFDL WG call 2007-11-28
by Mike Beckerle 27 Nov '07

27 Nov '07

Agenda for the call: 1) prefix length strings - as circulated in email 2) plan for contents of next draft I suggest this as a starting point: - binary/opaque should be resolved - null/defaults - facets discussion (which we use and how) integrated - choiceType enum + length properties on choice - prefix lengths - wildcards with dimensions - integer type added 3) Any other items ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

1 0

DFDL: Minutes from OGF WG call, 21 Nov 2007
by Ian W Parkinson 22 Nov '07

22 Nov '07

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 21 Nov 2007 Attendees Mike Beckerle (IBM) Geoff Judd (IBM) Steve Hanson (IBM) Suman Kalia (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) 1. Introduction Mike would like, in this meeting, to cover the hexBinary and base64Binary debate, and also to discuss the use of 'any' wildcards with minOccurs and maxOccurs. 2. 'Any' wildcards with minOccurs and maxOccurs The DFDL specification presently disallows the use of minOccurs and maxOccurs on 'any' wildcards, in contrast to XML schema. Simon stated that he sees no reason to forbid this, and that these properties might be useful, for example when structures could be followed by arbitrary extensions. Suman felt that the most common use case would be minOccurs="0" and maxOccurs="1". Mike wondered whether this would be useful within unordered groups, and thought it would be a good way to model an 'all' group containing some known fields and a number of unknown fields. Mike took an action item to investigate this further. 3. hexBinary and base64Binary The working group has been discussing, via email, the use of hexBinary and base64Binary types, along with 'enumeration' and 'pattern' properties. Mike asked the group whether we should disallow the 'enumeration' and 'pattern' properties on binary types (as they are difficult to use, in particular with base64Binary), and whether we should remove support for base64Binary (as it shares a value space with hexBinary, and is therefore a synonym of hexBinary in DFDL). Simon felt that this distinction remained a useful hint to any component emitting XML based on a DFDL infoset. He also observed that base64Binary is more commonly used than hexBinary, and is preferred. Mike argued that while base 64 might be preferred to hexadecimal in terms of space, hexadecimal is more readable. Steve observed that hexBinary is commonly used. While Mike felt that supporting fixed and default binary values might lead to requiring support for patterns and enumeration, the meeting agreed that there are use cases for default and fixed values - for example, some file formats use "eyecatchers" which are best expressed in hexadecimal. Mike pointed out that this could be achieved usng a string type, but suggested allowing 'default' and 'fixed' for hexBinary. Simon suggested that something similar would be necesssary for base64Binary, as some values (such as passwords and identifiers) are frequently expressed in base 64. Sandy Gao (IBM) has been asked to comment on whether there are any use cases where patterns are used with hexBinary of base64Binary. To conclude this discussion, Mike proposed the following: to retain support for both base64Binary and hexBinary, with identical content in DFDL; to allow both 'fixed' and 'default' for both base64Binary and hexBinary, but (pending further information from Sandy) to disallow 'pattern' and 'enum'. [Simon and Suman left the meeting] 4. Array Prefixes and Suffixes Mike asked whether the group was happy with the omission of array prefixes and suffixes. We know how to add these back should we ever need to, and there is a concern that including them would lead to many more array properties being necessary. Steve was happy with the present proposal. 5. Choice type and Length properties on Choice Mike observed that there is a need to distinguish between choice groups which are of constant length, and choice groups where the length is determined by the relevent subelement. Where the choice is unresolvable, it is not possible to have a choice of variable length. Steve felt that as we are able to make assertions, there would be very few cases where a choice is unresolvable. Mike pointed out that in an unresolvable choice group, each arm would need consistant enough syntax for a parser to be able to determine the end; and that this could be modelled as arms with enough information to discriminate. Geoff suggested that experience with IBM's MRM technology shows this to be unusual. The meeting considered two options. We could specify two properties, one to select between constant length and variable length; and one to select between resolvable and unresolvable. In this option, the combination variable-length/unresolvable would be disallowed. In the second option, we would have a single property with three possible values: constant length, variable length or unresolvable. The meeting agreed upon the second option. When experimenting with the DFDL language recently, Steve found specifying length on structures to be awkward. He proposed removing 'lengthKind' on choice elements, and Mike added that we would also wish to remove other associated properties such as 'intitator' and 'terminator'. On reflection, the meeting decided to keep these properties, noting that using these on a choice element is identical to wrapping the choice element in a sequence element with the same properties. 6. Other business There has, internally within IBM, been a discussion regarding length prefixes on strings. Mike will circulate a proposal to the working group, to allow prefix formats to be described through annotations on simpleType definitions. Meeting closed, 17:55 GMT Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL & length prefixes - proposal
by Mike Beckerle 21 Nov '07

21 Nov '07

For the broader DFDL workgroup (Martin, Simon, and any other onlookers), this proposal was circulated among the IBM DFDLers and we liked it, so we wanted others to take a look. ------------------------------------------------------------------------ Prefix lengths are common, but unfortunately there are many prefix length formats. For flexibility we want to be able to use a hidden field and calc properties, but this is clumsy enough to be very unattractive except for extreme cases. However, without it we have the keyword proliferation issue. Here's a way to get the best of both worlds. <element name="myString" type="string" dfdl:lengthKind="prefixed" dfdl:lengthPrefixType="packed4"/> here's the trick. Named types for length prefixes are just a real schema type definition of an integer type with dfdl annotations. . <simpleType name="packed4"> <restriction base="integer" dfdl:representation="decimal" dfdl:decimalType="packed" dfdl:numberFormat="9999" dfdl:decimalSigned="false" dfdl:prefixIncludesPrefixLength="false"/> </simpleType> The behavior is simple. Imagine that a hidden length prefix element is defined immediately before your element, and it's type is the one given here. You don't have to model that hidden element. DFDL knows how to do that for you. Just tell us it's type, including DFDL properties for the representation of that type. We could build in a few of these to some predefined profiles. E.g., dfdl:prefix2, and dfdl:prefix4 might be the typical 2 and 4 byte length prefix definitions you see in PL1 for example. I suggest the above because of the slippery slope where we have to gradually bifurcate all the integer properties to have lengthPrefix flavors of them e.g., lengthPrefixRepresentation = text or binary or decimal lengthPrefixLengthUnits = bits, bytes, or characters lengthPrefixLength = an integer lengthPrefixLengthIncludesPrefix (boolean) lengthPrefixTextNumberJustification lengthPrefixTextNumberFormat Anywhere you stop here there will be many cases that aren't covered.

1 0

DFDL wildcards with dimension
by Mike Beckerle 21 Nov '07

21 Nov '07

I did some investigation. 1) xs:any wildcards are not allowed inside xs:all groups at all. 2) The schema below is valid, and admits the instance document following it: <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <xsd:element name="x"> <xsd:complexType> <xsd:sequence> <xsd:element name="name"/> <xsd:element name="address"/> <xsd:any processContents="lax" minOccurs='0' maxOccurs='unbounded'/> </xsd:sequence> </xsd:complexType> </xsd:element> </xsd:schema> Here's the instance: <?xml version="1.0"?> <x> <name/> <address/> <foo/> <bar/> <quux/> <frobboz/> </x> 3) Note however, the only way to create an XML Schema which will validate both the instance above and the one below where the order is all scrambled, is an array-of-choice schema: <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <xsd:element name="x"> <xsd:complexType> <xsd:choice maxOccurs='unbounded'>  <xsd:element name="name"/> <xsd:element name="address"/> <xsd:any processContents="lax"/> </xsd:choice> </xsd:complexType> </xsd:element> </xsd:schema> This array-of-choice schema will accept this instance document also: <?xml version="1.0"?> <x> <foo/> <address/> <bar/> <name/> <quux/> <frobboz/> </x> The above array-of-choice schema obviously will also accept things where there are multiple name and address entries, which is undesirable, but the XPath expressions x/name and x/address will still work. <?xml version="1.0"?> <x> <address/> <address/> <foobar/> </x> Implications for DFDL: I suggest we should allow dimension on any wildcards because this unordered sequence DFDL fragment: <xsd:sequence dfdl:ordered="false"> <xsd:element name="name"/> <xsd:element name="address"/> <xsd:any id="unknown" processContents="lax" minOccurs='0' maxOccurs='unbounded'/> </xsd:sequence> describes something quite useful which otherwise requires an array of choice to model. Array-of-choice is generally undesirable as a model because it loses the cardinality information that says that name is required and address is required and there can be exactly 1 instance of each. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

2 1

Fw: OGF DFDL WG call today 2007-11-21
by Suman Kalia 21 Nov '07

21 Nov '07

Some interesting discussion on the hexBinary and base64Binary.. Schema authors also had reservations on having these 2 types with same value space and considered a possibility of having one binary type with encoding facet to specify hex and base64 which would have been a much better solution. I am equally perplexed on the use case for having pattern (and also enumeration) facet on the hexBinary and base64Binary as the facet work on the lexical space. May be the conservative approach for us is to treat these built-in schema type as is and only allow the dfdl set of annotations pertaining to allowable facets.. By copy to Sandy - do you know of a use case where pattern facet is used on hexBinary and base64Binary http://lists.w3.org/Archives/Public/www-xml-schema-comments/2001AprJun/0067… Some excerpts from discussion There are two possible ways to fix this before going to recommendation: 1. return to the CR status of a binary datatype with an encoding facet that could be either 'hex' or 'base64'. 2. eliminate one of hexBinary or base64Binary, and call the remaining type 'binary'. *Note that this is the most conservative decision*. The missing encoding could always be added back, but once released as an official recommendation parsers would be required to support both encodings for all time. Suman Kalia IBM Toronto Lab WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 11/21/2007 11:50 AM ----- Mike Beckerle <beckerle(a)us.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 11/21/2007 11:08 AM To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] OGF DFDL WG call today 2007-11-21 We may or may not achieve quorum today because of the US holiday tomorrow and big travel day today. I will join the call at 12noon US.ET, and if we have enough people by 12:05 then I'd like to discuss one or more of these topics * hexBinary and base64Binary - I still find these confusing. * array prefix and suffix - just review resolution of this issue - leaving out for now just to be conservative. Could put back in fairly easily. * choiceType and length properties on xs:choice My latest musings on hexBinary and base64Binary .... E.g., XSD allows pattern and enumeration facets on these <element name="aThing" type="base64Binary" length="3" pattern="AAAA|////" /> I think that pattern is a regexp for a base64 string matching 3 bytes of zeros or 3 bytes of all ones, but my regexp syntax is no doubt incorrectly escapified. ("A" is 6 bits of zero, "/" is 6 bits of 1 in base64). To me this is marvelously confusing. It feels downright silly to allow pattern and enumeration on these things. I'd like to adopt the strictest possible sensible thing. - hexBinary only, binary representation only, no pattern or enumeration facets supported or allowed for this type. There is still the issue of default/fixed. E.g., <element name="unknownStuff" type="hexBinary" fixed="F41306C0" dfdl:lengthKind="implicit" /> The user specifies what the bytes are as a hex string, and since they said it's hexBinary, they use hex to express the literal content. This is a way to say "right here in the data there's this blob of stuff I don't understand, but it always contains these data bytes". I've certainly seen the need for this sort of thing. It's being ignored on input, but on output it generates the fixed bytes of data so as to create valid output even when you don't understand this part of the data format. In fact, the cases I've seen are using this to skip over decimal numbers the wierd format of which isn't understood. The above pattern could be one or more decimal numbers in strange formats. However, I can see the slippery slope from here to allowing pattern and enumeration facets, so I'd be happy ruling out use of default and fixed on hexBinary type also, just to make it even simpler. Also, this element achieves the same end using our "%" escapes in strings. <element name="unknownStuff" type="string" dfdl:encoding="ascii" fixed="%F4%13%06%C0" dfdl:lengthKind="implicit"/> This works for any single-byte-wide character-set encoding. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

1 0

OGF DFDL WG call today 2007-11-21
by Mike Beckerle 21 Nov '07

21 Nov '07

We may or may not achieve quorum today because of the US holiday tomorrow and big travel day today. I will join the call at 12noon US.ET, and if we have enough people by 12:05 then I'd like to discuss one or more of these topics * hexBinary and base64Binary - I still find these confusing. * array prefix and suffix - just review resolution of this issue - leaving out for now just to be conservative. Could put back in fairly easily. * choiceType and length properties on xs:choice My latest musings on hexBinary and base64Binary .... E.g., XSD allows pattern and enumeration facets on these <element name="aThing" type="base64Binary" length="3" pattern="AAAA|////" /> I think that pattern is a regexp for a base64 string matching 3 bytes of zeros or 3 bytes of all ones, but my regexp syntax is no doubt incorrectly escapified. ("A" is 6 bits of zero, "/" is 6 bits of 1 in base64). To me this is marvelously confusing. It feels downright silly to allow pattern and enumeration on these things. I'd like to adopt the strictest possible sensible thing. - hexBinary only, binary representation only, no pattern or enumeration facets supported or allowed for this type. There is still the issue of default/fixed. E.g., <element name="unknownStuff" type="hexBinary" fixed="F41306C0" dfdl:lengthKind="implicit" /> The user specifies what the bytes are as a hex string, and since they said it's hexBinary, they use hex to express the literal content. This is a way to say "right here in the data there's this blob of stuff I don't understand, but it always contains these data bytes". I've certainly seen the need for this sort of thing. It's being ignored on input, but on output it generates the fixed bytes of data so as to create valid output even when you don't understand this part of the data format. In fact, the cases I've seen are using this to skip over decimal numbers the wierd format of which isn't understood. The above pattern could be one or more decimal numbers in strange formats. However, I can see the slippery slope from here to allowing pattern and enumeration facets, so I'd be happy ruling out use of default and fixed on hexBinary type also, just to make it even simpler. Also, this element achieves the same end using our "%" escapes in strings. <element name="unknownStuff" type="string" dfdl:encoding="ascii" fixed="%F4%13%06%C0" dfdl:lengthKind="implicit"/> This works for any single-byte-wide character-set encoding. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

1 0

Re: [DFDL-WG] Grammar as adapted to infoset
by Steve Hanson 21 Nov '07

21 Nov '07

Allowing dfdl:lengthKind on choices has other benefits. It means that elements, sequences and choices are consistent in how they are capable of being extracted (literal length, XPath, model determined, prefixed, null terminated). I don't think unresolvableWhenParsing implies fixed length storage. I could have an unresolvable choice being two variable length strings, terminated by the terminator of the choice. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle/Worcester/IBM@IBMUS 20/11/2007 21:23 To Steve Hanson/UK/IBM@IBMGB cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Rather than overloading "implicit" length to mean this, I think we need an enum-valued property for choices. Today there is this choiceUnresolvableWhenParsing boolean. I think this shoudl become choiceType="unresolvableWhenParsing". Then we can have an enum value for the case where all arms of the choice are assumed to be padded to max length of the alternative. So I think we have 3 enum values: variableLength fixedLength unresolvableWhenParsing variableLength means the storage taken depends on the alternative selected, Fixed length means shorter alternatives are padded to the length of the longest. unresolvable means what it means now, which is that you can't tell which variant it is (which also implies the storage required is fixed length). ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/19/2007 07:13 AM To Mike Beckerle/Worcester/IBM@IBMUS cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Or allow xs:choice to carry dfdl:length & dfdl:lengthKind. Then lengthKind="implicit" could mean the length is the longest of the modelled alternatives. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle/Worcester/IBM@IBMUS 16/11/2007 19:29 To Steve Hanson/UK/IBM@IBMGB cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset There are of course alternatives. E.g., a hidden or non-hidden field which is a byte array or hexBinary to absorb the padding bytes at the ends of the shorter variants. Or one could put trailing skip bytes on the last element or group of each of the shorter variants. One can also encapsulate the whole choice in a sequence whose length is the fixed maximum length. This is my personal favorite, since it uses the FinalUnused that is already in the grammar. E.g., suppose the length of the longest variant is 258 bytes. <sequence dfdl:lengthKind="explicit" dfdl:length="258" dfdl:lengthUnits="bytes" dfdl:applies="hereOnly"> <choice> ..... </choice> </sequence> The only thing I don't like about this is I don't have a way to express the length other than to hard-code the constant 258, but since tooling would typically put this in based on Cobol descriptors or other information I don't mind so much. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/16/2007 11:24 AM To Mike Beckerle/Worcester/IBM cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Hi Mike One extra thought post-call, caused by thinking about the FinalUnused part of a sequence. Some fixed length choices have a rule that each branch of the choice must be of equal length. This results in unused bytes for branches shorter than the maximum. In a bitstream instance, a user will see these bytes. Perhaps the grammar should acknowledge their existence, even if there are no DFDL properties to express the bytes (like FinalUnused)? Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle <beckerle(a)us.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 10/11/2007 00:16 To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] Grammar as adapted to infoset I took a crack at this. I ripped it out of the main document for faster review/turnaround. I think this matches the latest infoset email diagram. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Grammar as adapted to infoset
by Mike Beckerle 20 Nov '07

20 Nov '07

Rather than overloading "implicit" length to mean this, I think we need an enum-valued property for choices. Today there is this choiceUnresolvableWhenParsing boolean. I think this shoudl become choiceType="unresolvableWhenParsing". Then we can have an enum value for the case where all arms of the choice are assumed to be padded to max length of the alternative. So I think we have 3 enum values: variableLength fixedLength unresolvableWhenParsing variableLength means the storage taken depends on the alternative selected, Fixed length means shorter alternatives are padded to the length of the longest. unresolvable means what it means now, which is that you can't tell which variant it is (which also implies the storage required is fixed length). ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/19/2007 07:13 AM To Mike Beckerle/Worcester/IBM@IBMUS cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Or allow xs:choice to carry dfdl:length & dfdl:lengthKind. Then lengthKind="implicit" could mean the length is the longest of the modelled alternatives. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle/Worcester/IBM@IBMUS 16/11/2007 19:29 To Steve Hanson/UK/IBM@IBMGB cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset There are of course alternatives. E.g., a hidden or non-hidden field which is a byte array or hexBinary to absorb the padding bytes at the ends of the shorter variants. Or one could put trailing skip bytes on the last element or group of each of the shorter variants. One can also encapsulate the whole choice in a sequence whose length is the fixed maximum length. This is my personal favorite, since it uses the FinalUnused that is already in the grammar. E.g., suppose the length of the longest variant is 258 bytes. <sequence dfdl:lengthKind="explicit" dfdl:length="258" dfdl:lengthUnits="bytes" dfdl:applies="hereOnly"> <choice> ..... </choice> </sequence> The only thing I don't like about this is I don't have a way to express the length other than to hard-code the constant 258, but since tooling would typically put this in based on Cobol descriptors or other information I don't mind so much. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 Steve Hanson/UK/IBM@IBMGB 11/16/2007 11:24 AM To Mike Beckerle/Worcester/IBM cc dfdl-wg(a)ogf.org Subject Re: [DFDL-WG] Grammar as adapted to infoset Hi Mike One extra thought post-call, caused by thinking about the FinalUnused part of a sequence. Some fixed length choices have a rule that each branch of the choice must be of equal length. This results in unused bytes for branches shorter than the maximum. In a bitstream instance, a user will see these bytes. Perhaps the grammar should acknowledge their existence, even if there are no DFDL properties to express the bytes (like FinalUnused)? Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Mike Beckerle <beckerle(a)us.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 10/11/2007 00:16 To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] Grammar as adapted to infoset I took a crack at this. I ripped it out of the main document for faster review/turnaround. I think this matches the latest infoset email diagram. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046 -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

1 0

DFDL infoset and hexBinary
by Mike Beckerle 20 Nov '07

20 Nov '07

What is the DFDL infoset value for a hexBinary type element? Our doc says an element of type hexBinary has a value member in the infoset whose type is a hexBinary, but what is a hexBinary anyway? Consider this 12 bytes of data, dumped here as hex: 003100320033004100420043 If that's a string in UTF-16BE encoding, it looks like "123ABC" It is 12 bytes long. Suppose we describe this using this DFDL: <element name="a" type="hexBinary" length='12' /> As I understand it, this is correct XSD in that the length of a hexBinary is always specified as the number of 'octets' of the binary data, not characters. Now, the contents of the basic XML infoset for an element matching this schema are these 24 characters: "003100320033004100420043". This is because the XML infoset has no idea what this string is because it is not schema aware. An XML document corresponding to this might look like: <! xml version='1.0' encoding="UTF-16BE" !> <a>003100320033004100420043</a> So, the question is...... What are the contents of the DFDL Infoset for a hexBinary like this? Same 24 characters? or 12 'octets' of data? This is essentially the answer to the question "what IS a hexBinary?" or maybe "what is a hexBinary in the PSVI other than just a string?" ...mikeb

2 2