April 2012 - dfdl-wg - lists.ogf.org

Re: [DFDL-WG] String literal syntax for hexBinary ?? - Re: String literals - various usage patterns thereof
by Steve Hanson 19 Apr '12

19 Apr '12

Hi Tim Firstly, both your bulleted assertions are correct, but your conclusion is not. Secondly, let me flesh out my earlier reply about constructor functions. The last paragraph of Section 23 says: "The result of evaluating the expression must be a single atomic value of the type expected by the context, and it is a schema definition error otherwise". This is where the XPath constructors come into play. Eg: <element name="myHexBin" type="xs:hexBinary" dfdl:inputValueCalc="{ xs:hexBinary(...) }"/> These xs: constructors, plus the special fn:dateTime() constructor that DFDL adds, allow the correct types to be created. Note that you don't always need the constructors. An expression that returns a quoted value is returning an XPath string literal so that is automatically xs:string. An expression that returns an unquoted number is returning an XPath number literal so that can be xs:decimal, xs:integer, xs:double (depends whether the number contains a '.' or 'e' or 'E'). This is described here: http://www.w3.org/TR/xpath20/#id-literals So simply returning the literal 'DEADBEEF' will return an xs:string and if the context requires xs:hexBinary that is a schema definition error according to DFDL spec. A clarification is worth while though. Take the following expression: {if ../type eq 'A' then 10000 else 20000}. That returns xs:integer. - What if my context was xs:decimal? xs:integer is a restriction of xs:decimal so the value will always be in range, so is that 'auto-cast' allowed? - What if my context was xs:long or another restriction of xs:integer? The value may or may not be in range, so is that 'auto-cast' iff value in range? I think that we should auto-cast when type restrictoions are involved, and clarify that in the spec. We *could* change the spec to say that the result of the expression is always automatically cast to the type expected by the context. That takes some of the burden off the modeler and makes it much more likely that expressions written by XPath novices will return the correct results. But it could also hide accidental errors. Note this proposal I shall call (d) as it not the same as Mike's (c). If we made this change, then returning the literal 'DEADBEEF' for xs:hexBinary would succeed. I don't think it affects the desire for expressions to be statically type checkable - because it is known whether type X can be cast to type Y, so a cast mismatch can be statically detected. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org, Steve Hanson/UK/IBM@IBMGB Date: 19/04/2012 09:35 Subject: Re: [DFDL-WG] String literal syntax for hexBinary ?? - Re: String literals - various usage patterns thereof I'm pretty sure that the rules are: - DFDL expressions must not *contain* DFDL String Literals. They must be valid XPath 2.0 expressions except that the list of allowable function names includes the DFDL extension functions. - A DFDL expression is sometimes allowed to *return* a DFDL String Literal. In this case, the returned value is an xs:string that conforms to the DFDL String Literal syntax. But that does not apply to your example because the dfdl:inputValueCalc must return a value ( an XML value ) that is valid for the type of the element. I think that corresponds to your answer a) ; 'DEADBEEF' is a valid xs:hexBinary lexical value. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB Cc: dfdl-wg(a)ogf.org Date: 19/04/2012 07:42 Subject: [DFDL-WG] String literal syntax for hexBinary ?? - Re: String literals - various usage patterns thereof Sent by: dfdl-wg-bounces(a)ogf.org What is the DFDL string literal syntax for a hexBinary type value? E.g., I want a hex binary whose value is the 4 bytes described by this hex: DE AD BE EF. <element name="myHexBin" type="xs:hexBinary" dfdl:inputValueCalc="{ ... }"/> So, what can one syntactically put, for literal constant values, in the input value calculation expression? Note that this is legal pure (non-DFDL) XSD (I think) <element name="aHexBin" type="xs:hexBinary" fixed="DeadBeef"/> That is, the fixed/default are allowed and one specifies these values as just strings of hex digits. Notice no special escaping or anything. You just use a string that just so happens to contain hex digits. I think there are three possibilites (a) we allow "DEADBEEF" i.e., because the type of the expression is hexBinary, a string is cast to hexBinary by interpreting it as hex nibbles. (b) we require a special kind of string literal - a bytes-only string literal, so for example: "%#rDE;%#rAD;%#rBE;%#rEF;" is the way you create 4 bytes. If you just put characters, then that's a processing error - like a cast failure. Only raw-bytes allowed. (c) Anything you return from the expression is converted to a hexBinary by unparsing it to bytes (using current properties), then using the bytes as the hexBinary data. So you could have an expression that returns a double, and that would create 8 bytes if representation="binary". In this case the decimal number 3735928559 (hex 0xdeadbeef) as a binary bigEndian int would produce the 4 bytes I want. -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] dfdl-wg Digest, Vol 68, Issue 13
by Jeffrey C. Jacobs 18 Apr '12

18 Apr '12

Linking issue for "master" branch (branched from remotes/origin/master). When I try to compile I get a class not found error for ResourceResolver.class and LS$ResourceResolver.class. I guess I forgot to add a library to my Intellij project. Is this from xerces? Just want to be sure I link against the right library. Thanks. Jeffrey.

1 0

Fw: Artifact changed: Example set of DFDL 1.0 properties
by Steve Hanson 18 Apr '12

18 Apr '12

This document is now in Public Comment. Please review. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 18/04/2012 09:29 ----- From: SourceForge Administrator <ogf-it(a)ogf.org> To: Steve Hanson/UK/IBM@IBMGB Date: 17/04/2012 17:42 Subject: Artifact changed: Example set of DFDL 1.0 properties artf6549: Example set of DFDL 1.0 properties Modified on 04/17/2012 by Joel Replogle Comment was added: Apologies for the delay. This entered 30-day public comment on 2012-04-17. Comment URL: http://www.ogf.org/gf/docs/comment.php?id=387 Respond by visiting: https://forge.ogf.org/sf/go/artf6549 --------------------------------------------------------------------- Artifact details:- Project: Editor Tracker: Submit OGF Draft Group: Data Status: Public Comment Period Category: Informational Priority: 3 Assigned To: Joel Replogle resolution: Description: This informational document provides an example to the OGF community of the Data Format Description Language (DFDL) standard. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL String Literals - diagram
by Mike Beckerle 17 Apr '12

17 Apr '12

I was thinking about how to implement DFDL string literals. I went through the spec, and created a diagram which shows in a quasi-UML-class-diagram, what I think is the currently specified behavior of String literals and the properties that have them as their values. The diagram is attached. If this shows with black or other background instead of white, look for a setting on whatever you are using to view the png file. I had to set mine to white background as the default was an unreadable checkerboard. Some of the "classes" on here represent combinations of properties e.g., NilKindLiteralValueTextRepSpec means nilKind='literalValue' and representation="text". I think it is worth it to review this and see if we agree generally with what is allowed to specify a raw-byte entity and what is not, etc. Question: Can DFDL Regular Expressions include DFDL Entities? If so which kinds? I would assume (and in the drawing) that is only plain and numeric character entities, not raw bytes, and not character class entities. ...mikeb -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

1 0

Invitation: OGF DFDL Working Group extra call (20 Apr 14:30 GDT in IBM Hursley DM2046)
by Steve Hanson 17 Apr '12

17 Apr '12

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl GridForge DFDL: http://forge.ogf.org/projects/dfdl-wg/

1 0

OGF DFDL WG Call Agenda 2012-04-17 - extra call needed
by Steve Hanson 17 Apr '12

17 Apr '12

Call was held but only action 145 was discussed. We need to progress the other actions and agenda items, so I will schedule an extra call for this Friday. Please find agenda for the above call on GridForge at https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.dfdl-wg/… . Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Agenda 2012-04-17
by Steve Hanson 16 Apr '12

16 Apr '12

Please find agenda attached for the above call on GridForge at https://forge.gridforum.org/sf/docman/do/downloadDocument/projects.dfdl-wg/… . Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] Fw: Action 145: 'dispatch' way of discriminating a choice for better performance
by Steve Hanson 16 Apr '12

16 Apr '12

Hi Tim, I think all our expressions today return a schema logical type, so you are correct in that the expression should not return DFDL string literal. But I don't see the harm in letting dfdl:elementID be a DFDL string literal. We can disallow some of the DFDL entities, like we do for padding character, etc, if they don't make sense. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM To: Steve Hanson/UK/IBM@IBMGB Cc: dfdl-wg(a)ogf.org Date: 11/04/2012 10:15 Subject: Re: [DFDL-WG] Fw: Action 145: 'dispatch' way of discriminating a choice for better performance I think the type of the property should be 'xs:string', not 'DFDL String Literal'. A DFDL String Literal allows raw bytes, DFDL character entities and 'generic mnemonics' such as '%WSP*'. I don't see any value in allowing those in this property - the data in the data stream can easily be converted to a string as part of the DFDL expression. If we did allow DFDL String Literal then we would have to specify exactly how to match one DFDL String Literal against another, but I don't think we should go there. Proposed new wording: A new dfdl:element property is added called dfdl:elementID of type 'DFDL String Literal' 'xs:string'. This provides a single alternative name for the element. Only allowed on local element and global element, not on element reference or simple type. A new dfdl:choice property is added called dfdl:choiceBranchRef of type 'DFDL Expression'. The expression must evaluate to a DFDL String Literal xs:string. The resultant string must match (case insensitive) the dfdl:elementID property values of one of the element branches of the choice, and if so discriminates in favour of that branch. The parser then goes straight to that branch, ignoring schema order. Because the branch is 'known to exist' no backtracking takes place if a processing error subsequently occurs. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 From: Steve Hanson/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org Date: 11/04/2012 03:57 Subject: [DFDL-WG] Fw: Action 145: 'dispatch' way of discriminating a choice for better performance Sent by: dfdl-wg-bounces(a)ogf.org Updated to reflect the outcome of DFDL WG call on 5th April. Revised details: A new dfdl:element property is added called dfdl:elementID of type 'DFDL String Literal'. This provides a single alternative name for the element. Only allowed on local element and global element, not on element reference or simple type. A new dfdl:choice property is added called dfdl:choiceBranchRef of type 'DFDL Expression'. The expression must evaluate to a DFDL String Literal. The resultant string must match (case insensitive) the dfdl:elementID property values of one of the element branches of the choice, and if so discriminates in favour of that branch. The parser then goes straight to that branch, ignoring schema order. Because the branch is 'known to exist' no backtracking takes place if a processing error subsequently occurs. Rules: - Both properties behave like dfdl:ref and dfdl:hiddenGroupRef in that it is not possible to set a value in scope by a dfdl:format annotation, and is only set at its point of use. This is because there is nothing sensible that could be set in scope **. Empty string is not an allowed value. - Both properties are only used when parsing. - When dfdl:choiceBranchRef is present, all choice branches must be local elements or element references. It is a schema definition error otherwise. - It is a processing error if the resolved value of dfdl:choiceBranchRef does not match one of the branches. - It is a schema definition error if individual dfdl:elementID values are not unique across all elements that are branches of an xs:choice that carries dfdl:choiceBranchRef. - It is not a schema definition error if both dfdl:initiatedContent and dfdl:choiceBranchRef, are provided on the same choice, nor if a discriminator exists on that choice branch. ++ ** Consequently adding support for the property to existing DFDL implementations will not suddenly cause errors to appear in existing DFDL schemas. ++ Reasoning. If a discriminator is evaluated and there is no point of uncertainty, it simply behaves like an assert, specifically if it fails it causes a processing error without backtracking. Use of dfdl:choiceBranchRef is just another way to get into this situation. (The property dfdl:initiatedContent may be rewritten as a series of discriminators on the child elements which test that the (initiated) element exists, hence same applies). Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 28/03/2012 16:32 ----- From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org Date: 27/03/2012 14:09 Subject: Fw: [DFDL-WG] Action 145: 'dispatch' way of discriminating a choice for better performance Hi Mike I agree. When I wrote the SWIFT example expression that I realised this was becoming a bit too complicated. IBM MRM has a quick way of dispatching a choice that uses a similar mechanism to the one you propose. It has an optional property called 'Message Alias'. The parser first tries to match the tag against the element name, and if that fails it tries the alias. I've modified my proposal accordingly. Regarding wildcards, the approach you show was what we had agreed upon a couple of years ago when we dropped wildcards. I would prefer not to add them back in for 1.0, but I have had some feedback from IBMers modeling envelope/payload formats that they want a loose-coupling approach as that fits better with other tools and is scalable. Let's treat this as a separate issue. I'll go back to the IBMers so that I fully understand their concerns, and if necessary I will bring back to the WG as a new agenda item. So all we need to ensure for now is that whatever mechanism we come up with is extensible to wildcards, if needed. Revised details: A new dfdl:element property is added called dfdl:elementAlias of type 'List of String'. This provides one or more alternative names for the element. Only allowed on local element and global element, not on element reference or simple type. A new dfdl:choice property is added called dfdl:choiceBranchRef of type DFDL Expression. The expression must evaluate to String. The string must match (case insensitive) either the element name or one of the dfdl:elementAlias property values of one of the element branches of the choice, and if so discriminates in favour of that branch. The parser then goes straight to that branch, ignoring schema order. Because the branch is 'known to exist' no backtracking takes place if a processing error occurs. Rules: - Both properties behave like dfdl:ref and dfdl:hiddenGroupRef in that it is not possible to set a value in scope by a dfdl:format annotation, and is only set at its point of use. This is because there is nothing sensible that could be set in scope. But it has the benefit that adding support for the property to existing DFDL implementations will not suddenly cause errors to appear in existing DFDL schemas. Empty string is not an allowed value. - Both properties are only used when parsing. - When dfdl:choiceBranchRef is preent, all choice branches must be local elements or element references. It is a schema definition error otherwise. - It is a processing error if the resolved xs:string value of dfdl:choiceBranchRef does not match one of the branches. - It is a schema definition error if a choice has both dfdl:choiceBranchRef set and dfdl:initiatedContent="yes". - It is a schema definition error if individual dfdl:elementAlias values are not unique a) across all elements that are branches of a given xs:choice and b) across all global elements in the schema. So we now have the ability to apply a simple lookup before we start to process a choice. That makes the processing time for each branch of the choice independent of its order in the schema. Questions: - What happens if we encounter a discriminator once we are processing the branch and its point of uncertainty is the choice ? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 27/03/2012 13:41 ----- From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB Cc: dfdl-wg(a)ogf.org Date: 21/03/2012 15:57 Subject: Re: [DFDL-WG] Action 145: 'dispatch' way of discriminating a choice for better performance (updated) Sent by: dfdl-wg-bounces(a)ogf.org The whole point of this thing is to be faster, not more general, so my reaction is too much XPath expression complexity here. Consider this. If the tag is some 2-character code that does NOT want to be the same as the element names (for example because they're digits, so they can't be the element names exactly since element names have to begin with an alpha char. Digits also aren't useful from readability perspective as names), then you'll need a big lookup table in the choiceBranchRef expression that translates from the codes to the QNames. We don't have a case statement in the expression language, so you've just moved the big linear evaluation chain out of evaluating choice discriminators one after another and into a big if-then-else nest in the choiceBranchRef expression. I don't see a performance gain here. I suggest dropping the QName stuff, and requiring a dfdl:choiceID property on the elements that is an NCName. (Well, we might want those QName functions anyway in the expression language. But I wouldn't use them for this rapid choice dispatch feature. You could certainly use them in discriminators. ) The expression would then have to evaluate to a value that is matched against this choiceID. I suggest exact match, not respecting ignoreCase for example. That eliminates all the QName complexity and is amenable to high-speed compact lookup table implementation. I tend to think the element names want to be a little bit more descriptive than these tag values would want to be so using the element names as the tags feels undesirable to me. Particularly because we want the tags to be conveniently computed, for example by just grabbing a fixed-length string out of a data field. You end up with something like this: <element name="tag" type="string" dfdl:length="{ 2 }" ..../> <choice dfdl:choiceBranchRef="{ ../tag }"> <element name="someName" dfdl:choiceID="02" .../> <element name="anotherName" dfdl:choiceID="73" .../> .... </choice> As for the wild-card issue. I think we can finesse this. Consider this model: <element name="tag" type="string" dfdl:length="{ 2 }" ..../> <choice>  <choice dfdl:choiceBranchRef="{ ../tag }"> <element name="someName" dfdl:choiceID="02" .../> <element name="anotherName" dfdl:choiceID="73" .../> .... </choice>  <element name="extensionRecord"> <complexType> <sequence>  <element name="extensionType" type="string" dfdl:inputValueCalc="../../tag" .../> .... </sequence> </complexType> </element> </choice> The inner choice uses the fast dispatch. The outer choice lets me also have an alternative that absorbs a more general syntax to provide some way for a user to model their extensions to the choice set. The extensionType field captures the tag and stores it inside the extension record where it won't get disassociated. The user's "extensionRecord" would not be a special DFDL wildcard construct, just an element format they create which is general enough to parse their extensions. Or this extension could be predefined as part of a standard, to accept any standard-defined acceptable extension record a user might need so long as there is some set of rules all extension records must obey. Given this, do we really need special wildcard constructs? ...mikeb From: Steve Hanson <smh(a)uk.ibm.com> To: dfdl-wg(a)ogf.org, Date: 24.03.2012 02:27 Subject [DFDL-WG] Action 145: 'dispatch' way of discriminating a choice : for better performance The enveope/payload style of data format is quite common, where the envelope provides control information and the payload contains the business data. Examples are SWIFT and SAP IDocs. Typically the envelope contains a tag that identifies the payload, which can be one of many types. For SWIFT there are 300 possible types. To model this today in DFDL requires an xs:choice with each type modeled as an xs:element branch of the choice. A discriminator on each xs:element refers back to the envelope tag element thus enabling the choice to be resolved. There are two issues with this approach. 1) Performance. Even if the elements in the branches are ordered for expected frequency, there will still be cases when tens or hundreds of discriminators need to be evaluated before the choice is resolved. 2) Tight coupling. When a new type is added, a new element branch needs to be added to the choice. Action 145 proposes a mechanism to solve issue #1 and which opens the door to a possible extension to DFDL to solve issue #2 - namely a faster way to resolve a choice. Details: A new dfdl:choice property is added called dfdl:choiceBranchRef of type DFDL Expression. The expression must evaluate to a QName which corresponds to one of the element branches of the choice, and asserts 'known to exist' for that branch. Rules: - The property behaves like dfdl:ref and dfdl:hiddenGroupRef in that it is not possible to set a value in scope by a dfdl:format annotation, and is only set at its point of use. This is because there is nothing sensible that could be set in scope. But it has the benefit that adding support for the property to existing DFDL implementations will not suddenly cause errors to appear in existing DFDL schemas. - Empty string is not an allowed value. - The property is only used when parsing. - All branches must be local elements or element references. It is a schema definition error if any branch is a sequence, a choice or a group reference. - It is a processing error if the QName does not resolve to one of the branches when parsing.. - It is a schema definition error if a choice has the property set and also has dfdl:initiatedContent="yes" set locally. - Because the expression must return a QName, the expression language must provide a constructor for creating a QName from a namespace string and a name string. If you take SWIFT MT103 payload as an example, the tag in the envelope says "103" but a DFDL schema would actually model the global MT103 element with name "Document" and namespace ="urn:swift:xsd:fin.103.2011". So the dfdl:choiceBranchRef expression would have to look like: {fn:QName(fn:concat(fn:concat('urn:swift:xsd:fin.', FinMessage/Block2/MessageType), ".2011"), 'Document')} So we now have the ability to derive a QName and apply it before we start to process a choice. That makes the processing time for each branch of the choice independent of its order in the schema. We still have issue #2 so when a new payload is added, a new branch must be added to the choice. A solution to this is to allows xs:any wildcard elements back into DFDL, then provide a property dfdl:wildcardRef which works in the same way as dfdl:choiceRef. So at the point of encountering the wildcard we know its resolution in the schema. This obviously will require some further discussion, but you can see how this ability to evaluate an expression and return a QName can be used in multiple ways. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg

1 0

Errata 2.22 - revision
by Steve Hanson 13 Apr '12

13 Apr '12

Errata 2.22 says that textStandardDecimalSeparator must be set if pattern contains a '.' but it must also be set if the pattern contains 'E' or '@' as these on their own can cause a decimal separator to be output (due to use of significant digits logic). Errata needs to be updated to reflect this. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

How to choose the correct choice branch when serializing
by Tim Kimber 13 Apr '12

13 Apr '12

There is an interesting edge case which arises when the serializer encounters a choice group. A DFDL xsd is structured as follows: <root> <choice> <sequence> <firstname/> <lastname/> <postcode/> </sequence> <sequence> <lastname/> <telephoneNumber/> </sequence> </choice> </root> Note that both branches of the choice are sequences, not elements. The infoset is <root> <lastName/> <telephoneNumber/> </root> The likely action of the serializer is: - pick the first branch of the choice ( because it contains lastname ) - output the default value of firstname ( assuming that firstname has minOccurs = 1 ) - output lastname - issue a processing error because lastname is not present in the info set ( assuming that lastname has minOccurs = 1 ). ...but the user might have intended this: - reject the first branch of the choice because there is a missing required field - select the second branch of the choice and successfully process the entire info set The DFDL specification does not state what the behaviour should be. I think the options are: a) state explicitly that the serializer will choose the first branch that contains a matching element, regardless of minOccurs b) invent a new rule that causes the parser to back out of a branch and try another branch if there is a minOccurs error while processing the branch c) disallow sequences and choices as immediate children of a choice group Currently I'm leaning toward a) by process of elimination, for the following reasons: b) would make this scenario work, but I think it would impose a lot of work on implementers because it would require the serializer to do backtracking. c) would simplify a lot of things, but I think it's too restrictive - I can imagine complex data formats where is might be useful to have a choice as the direct child of a choice because the discrimination rules might be easier to express in a two-level structure. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

3 2