February 2013 - dfdl-wg - lists.ogf.org

Implementation limits
by Steve Hanson 11 Jul '13

11 Jul '13

Just checking - did we close on this? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 21/01/2013 18:38 ----- From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: Steve Hanson/UK/IBM@IBMGB, Cc: dfdl-wg(a)ogf.org Date: 19/11/2012 21:08 Subject: Re: [DFDL-WG] proposed clarification/narrowing - no twos-complement decimal, integer or nonNegativeInteger Do we have a section where this goes? There are lots of implied implementation limits. Maximum string length? Maximum hexbinary length? Maximum number of packed digits in various numbers. Maximum length of a delimiter? Maximum length of a fixed attribute? Maximum length of a pattern regex expression? Maximum length of a regular expression? There's also limits on buffering of data for unparsing, etc. There's also limits on lookahead/speculation. I.e., how far an implementation is willing to speculate forward. etc. For some types these are clear: e.g., binary xs:int is 32 bits max. Do we need a numbered section on implementation specific limits. Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

3 4

Representing multiple spaces
by Garriss Jr., James P. 01 Mar '13

01 Mar '13

Suppose I have a terminator that can be multiple spaces, whether 0 spaces, 1 space, 2 spaces, or more spaces. No other types of whitespace allowed, just spaces. Because there's this entity: %WSP*; I assumed there would also be this entity: %ES*; But there's not. Why not? How would I represent this terminator? TIA

3 5

Alternate Initiators?
by Garriss Jr., James P. 27 Feb '13

27 Feb '13

Consider the following IMF headers: Message-ID: <CACQ76nh2VDypuwm1aHc1-s8ZmmyXg8XEosS6af6yurkVdT+8QQ(a)mail.gmail.com> Message-Id: <208E8372-A9D9-468A-9B4B-BD97ED28F64C(a)garriss.org> Notice the difference: Message-ID: vs Message-Id: Per the RFC, only the first is legal, but both are used in practice, and we'd like to allow either. In our DFDL schema, "Message-ID" is an initiator, but suddenly we have two alternate initiators for the same element. How we model this in DFDL? TIA

3 3

OGF DFDL WG Call Minutes 2013-02-26
by Steve Hanson 26 Feb '13

26 Feb '13

Please find minutes from the above call on Redmine at http://redmine.ogf.org/dmsf_files/13012?download= Errata document is now at v011.3 here: http://redmine.ogf.org/dmsf_files/8104?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Agenda 2013-02-26
by Steve Hanson 25 Feb '13

25 Feb '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13012?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

How to parse an email address
by Garriss Jr., James P. 25 Feb '13

25 Feb '13

Suppose I want to model an email address: dfdl-wg(a)ogf.org It's a sequence of a string, an a character (@), and a string. In the MBTK it might look like this: [cid:image001.png@01CE10EC.6600AE50] The problem, of course, is that the parser will grab all of "dfdl-wg(a)ogf.org" instead of just "dfdl-wg". Setting validation facets doesn't help, b/c validation happens after parsing. Setting @ as a separator or terminator would work, but that doesn't seem right either, as the @ is part of the email address, and I would want to keep it in the infoset. How do I think about this problem? TIA

4 4

Clarify behaviour of an empty xs:choice
by Steve Hanson 25 Feb '13

25 Feb '13

The DFDL spec says that all branches in an xs:choice must not have minOccurs '0' and if all branches are processed without a match then it is a processing error. But the DFDL spec also allows the creation of an empty xs:choice, that is, a choice without any branches at all. Some questions arise: 1) What happens when choiceLengthKind is 'explicit' - SDE ? Insist that choiceLength is '0' ? 2) What happens if choiceBranchRef is specified ? 3) Assuming any syntax matches, an empty choice always parses successfully. Read on... I have been using an empty choice for modeling data formats where user-defined extensions are permitted. This was the mechanism agreed upon by the WG to model this scenario in the absence of xs:any support in DFDL 1.0. Here's an example from a schema that models the TLOG retail format: <xsd:choice> <xsd:element ref="TransactionRecord00"/> <xsd:element ref="TransactionRecord01"/> <xsd:element ref="TransactionRecord04"/> <xsd:element ref="TransactionRecord05"/> <xsd:element ref="TransactionRecord98"/> <xsd:group ref="choice_Custom"/> </xsd:choice>  <xsd:group name="choice_Custom"> <xsd:choice>  </xsd:choice> </xsd:group> The intent is that the user adds his own records in the choice_Custom group. If an unknown record is parsed that fails to match any of the pre-defined records and any of the user-defined records, it will cause a processing error. However that didn't actually work. Because choiceCustom is initially empty, it will parse successfully against any input and in doing so discriminate the outer choice, meaning the processing error is not thrown. As soon as an element is added to choice_Custom, then it behaves as originally intended - an unknown record causes a processing error in choice_Custom, so the outer choice never discriminates. To work around this I added an assert to the choice to ensure it fails when empty.  <xsd:group name="choice_Custom"> <xsd:choice>   <xsd:sequence> <xsd:annotation><xsd:appinfo source=" http://www.ogf.org/dfdl/"> <dfdl:assert>{fn:false()} </dfdl:assert> </xsd:appinfo></xsd:annotation> </xsd:sequence> </xsd:choice> </xsd:group> So is an empty choice a genuinely useful construct to allow? It didn't actually help my scenario in 3). Are there any where it would make sense? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

OGF DFDL WG Call Minutes 2013-02-19
by Steve Hanson 19 Feb '13

19 Feb '13

Please find minutes from the above call on Redmine at http://redmine.ogf.org/dmsf_files/13007?download= Errata document is now at v011.3 here: http://redmine.ogf.org/dmsf_files/8104?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] representation properties vs. format properties
by Steve Hanson 19 Feb '13

19 Feb '13

I'd like to close this out on the next WG call please. Recommendation is below. 202 Term 'representation property' (All) 29/1: Two kinds of format property, how best to distinguish them in the spec. 5/2: Continued the discussion around 'representation' and 'format'. As part of Steve's review of the spec he will look at this issue and make a recommendation. 12/2: Steve has made a recommendation. Also has recommended that the DFDL spec uses 'property' instead of 'attribute' when referring to XSD properties like minOccurs (technically they are properties with an atribute representation). Review for next call. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org, Cc: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> Date: 12/02/2013 14:20 Subject: Re: [DFDL-WG] representation properties vs. format properties I have looked at this as part of my recent review of the updated DFDL spec. My conclusions: - The term 'DFDL property' applies to all DFDL properties carried on any DFDL annotation. We should use it in preference to 'attribute' even when an XML attribute is the only way that a DFDL property may be specified. (This is in keeping with XML Schema, where for example the 'name' property of a component is given by an XML attribute representation 'name'). - Similarly, the term 'XSD property' applies to all XML Schema properties, such as default, fixed, minOccurs, maxOccurs, etc. - The term 'format property' applies to all DFDL properties carried on DFDL format annotations. (One could envisage similar terms 'statement property' and 'defining property' but we don't need them at the moment). - The term 'representation property' applies to all format properties that are used to describe grammar regions. The revised grammar uses 'Rep' in grammar terms like 'SimpleElementEmptyRep' etc. - The term 'non-representation property' applies to all format properties that are not representation properties, specifically dfdl:ref, dfdl:hiddenGroupRef, dfdl:elementId, dfdl:choiceBranchRef, dfdl:inputValueCalc, dfdl:outputValueCalc, dfdl:escapeSchemeRef. The one non-standard property is dfdl:escapeSchemeRef, because it is a non-representation property yet can still be placed into scope. All the other non-representation properties can not be placed into scope. I am ok with that, it just means that whether a format property is a non-representation property is decoupled from whether it can be placed into scope or not, and should be considered on a case-by-case basis. Alternatively we can say that dfdl:escapeSchemeRef is a representation property even though it points at a defining annotation (and so is similar in this respect to dfdl:prefixLengthType). - Property dfdl:representation must be qualified by the namepace prefix when referred to in the spec, and preferably either preceded by 'property' or used on its own, rather than followed by 'property', to avoid any ambiguity. The term 'attribute' should be used only when describing how a property is represented in XML terms. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Date: 29/01/2013 10:12 Subject: Re: [DFDL-WG] representation properties vs. format properties We do have an actual problem in the spec, as the term 'representation property' is being used to mean both any DFDL property (other than 'ref') and also the property 'representation'. This is best seen in Table 13.1. However I am only seeing one example of the term 'format property' - in section 24 ? Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: dfdl-wg(a)ogf.org, Date: 28/01/2013 18:27 Subject: [DFDL-WG] representation properties vs. format properties Sent by: dfdl-wg-bounces(a)ogf.org We use both the terms representation property and format property extensively. Do we want to declare these to mean the same thing as each other, or globally replace one with the other. -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 2

OGF DFDL WG Call Agenda 2013-02-19
by Steve Hanson 18 Feb '13

18 Feb '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/13007?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0