- dfdl-wg - lists.ogf.org

Re: [DFDL-WG] Proposed spec errata for calendarPattern I and T symbols
by Steve Hanson 16 Jan '12

16 Jan '12

Tim, If a user wants a calendar type in his infoset for a 4 digit year, the recommendation would be to use a calendar pattern of 'YYYY'. The missing components get defaulted to values from the Unix epoch. That's much easier than using inputValueCalc (which would only be necessary if he wanted values other than the Unix epoch). Bearing that in mind, allowing the 'Year' and 'Year and month' granularities is not unnatural. On balance I think it is easier just to say that DFDL 'I' symbol supports all the granularities, and therefore all the W3C ISO8601 profile. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Tim Kimber/UK/IBM To: Steve Hanson/UK/IBM@IBMGB Cc: dfdl-wg(a)ogf.org, dfdl-wg-bounces(a)ogf.org Date: 16/01/2012 10:00 Subject: Re: [DFDL-WG] Proposed spec errata for calendarPattern I and T symbols I would be in favour of the behaviour listed under 'Alternatives'. If DFDL doesn't support gYear, gYearMonth etc then DFDL users should model the various parts of the type as integers/strings. They can always use XPath constructors with InputValueCalc if they really need to handle these types as calendar types. regards, Tim Kimber, Common Transformation Team, Hursley, UK Internet: kimbert(a)uk.ibm.com Tel. 01962-816742 Internal tel. 246742 From: Steve Hanson/UK/IBM@IBMGB To: dfdl-wg(a)ogf.org Date: 16/01/2012 05:05 Subject: [DFDL-WG] Proposed spec errata for calendarPattern I and T symbols Sent by: dfdl-wg-bounces(a)ogf.org For discussion on next WG call. Section 13.11.1. of the DFDL 1.0 spec defines the formatting symbols that can be used for calendar patterns. In addition to the supported ICU symbols, it also permits some extra symbols (I and T) that are used as a shorthand for ISO8601 format date/times. The following is quoted from the spec. Currently Symbol Meaning Presentation Example I ISO8601 Date/Time (Text) 2006-10-07T12:06:56.568+01:00 IU ISO8601 Date/Time (Text) 2006-10-07T12:06:56.568Z with output "Z" if the time zone is +00:00) T ISO8601 Time (Text) 12:06:56.568+01:00 (up to HH:mm:ss.SSSZZZ) TU ISO8601 Time (Text) 12:06:56.568Z (similar to T, but a time zone of +00:00 is replaced with 'Z') The 'I' and 'T' pattern characters should be used on their own to format dates and times which match the following subset of the ISO8601 standard. · The restricted profile as proposed by the W3C at http://www.w3.org/TR/NOTE-datetime · Truncated representations of calendar dates, as specified in section 5.2.1.3 of ISO8601:2000 o Basic format (subsections c, e, and f) o Extended format (subsections a, b, and d) An analysis shows that this definition of I and T is both incomplete and overly complex. The restricted W3C profile only defines the format for date/time and date-only, there is no time-only format The restricted W3C profile supports 6 'granularities' and states that referencing standards must say which they support The restricted W3C profile states that referencing standards must be explicit as to the number of fractional seconds supported The support for truncated representations of calendar dates was added to support xs:gYear, xs:gYearMonth, etc, which are not supported in 1.0 It does not say what the canonical output 'granularity' is. Proposal Here is what is proposed to correct this. Note the T symbol is dropped. It was introduced to allow xs:dateTime to expect just a time, but that is not necessary (it was copied from IBM MRM). I've stated any alternatives that we can consider at the end. Symbol Meaning Presentation Example I ISO8601 Date/Time (Text) 2006-10-07T12:06:56.568+01:00 IU ISO8601 Date/Time (Text) 2006-10-07T12:06:56.568Z with output "Z" if the time zone is +00:00) The 'I' symbol must not be used with any other symbol other than the 'escape for text' symbol. It represents calendar formats that match those defined in the restricted profile of the ISO8601 standard proposed by the W3C at http://www.w3.org/TR/NOTE-datetime.The formats are referred to as 'granularities'. xs:dateTime. When parsing, the data must match one of the granularities. When unparsing, the fullest granularity is used. xs:date. When parsing, the data must match one of the date-only granularities. When unparsing, the fullest date-only granularity is used. 'IU' is permitted for xs:date but the 'U' is effectively ignored as there is no time zone in the date-only granularities. xs:time. When parsing, the data must match only the time components of one of the granularities that contains time components. When unparsing, the time components of the fullest granularity are used. The literal 'T' character is not expected in the data when parsing and is not output when unparsing. The number of fractional second digits supported is implementation dependent but must be at least one. For a granularity that omits components, when parsing the values for the omitted components are supplied from the Unix epoch 1970-01-01T00:00:00.000 . Alternatives: As above but don't support the first two granularities ('Year', 'Year and month') on the grounds that they are really matching xs:gYear and xs:gYearMonth. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Proposed spec errata for calendarPattern I and T symbols
by Steve Hanson 16 Jan '12

16 Jan '12

For discussion on next WG call. Section 13.11.1. of the DFDL 1.0 spec defines the formatting symbols that can be used for calendar patterns. In addition to the supported ICU symbols, it also permits some extra symbols (I and T) that are used as a shorthand for ISO8601 format date/times. The following is quoted from the spec. Currently Symbol Meaning Presentation Example I ISO8601 Date/Time (Text) 2006-10-07T12:06:56.568+01:00 IU ISO8601 Date/Time (Text) 2006-10-07T12:06:56.568Z with output "Z" if the time zone is +00:00) T ISO8601 Time (Text) 12:06:56.568+01:00 (up to HH:mm:ss.SSSZZZ) TU ISO8601 Time (Text) 12:06:56.568Z (similar to T, but a time zone of +00:00 is replaced with 'Z') The 'I' and 'T' pattern characters should be used on their own to format dates and times which match the following subset of the ISO8601 standard. · The restricted profile as proposed by the W3C at http://www.w3.org/TR/NOTE-datetime · Truncated representations of calendar dates, as specified in section 5.2.1.3 of ISO8601:2000 o Basic format (subsections c, e, and f) o Extended format (subsections a, b, and d) An analysis shows that this definition of I and T is both incomplete and overly complex. The restricted W3C profile only defines the format for date/time and date-only, there is no time-only format The restricted W3C profile supports 6 'granularities' and states that referencing standards must say which they support The restricted W3C profile states that referencing standards must be explicit as to the number of fractional seconds supported The support for truncated representations of calendar dates was added to support xs:gYear, xs:gYearMonth, etc, which are not supported in 1.0 It does not say what the canonical output 'granularity' is. Proposal Here is what is proposed to correct this. Note the T symbol is dropped. It was introduced to allow xs:dateTime to expect just a time, but that is not necessary (it was copied from IBM MRM). I've stated any alternatives that we can consider at the end. Symbol Meaning Presentation Example I ISO8601 Date/Time (Text) 2006-10-07T12:06:56.568+01:00 IU ISO8601 Date/Time (Text) 2006-10-07T12:06:56.568Z with output "Z" if the time zone is +00:00) The 'I' symbol must not be used with any other symbol other than the 'escape for text' symbol. It represents calendar formats that match those defined in the restricted profile of the ISO8601 standard proposed by the W3C at http://www.w3.org/TR/NOTE-datetime.The formats are referred to as 'granularities'. xs:dateTime. When parsing, the data must match one of the granularities. When unparsing, the fullest granularity is used. xs:date. When parsing, the data must match one of the date-only granularities. When unparsing, the fullest date-only granularity is used. 'IU' is permitted for xs:date but the 'U' is effectively ignored as there is no time zone in the date-only granularities. xs:time. When parsing, the data must match only the time components of one of the granularities that contains time components. When unparsing, the time components of the fullest granularity are used. The literal 'T' character is not expected in the data when parsing and is not output when unparsing. The number of fractional second digits supported is implementation dependent but must be at least one. For a granularity that omits components, when parsing the values for the omitted components are supplied from the Unix epoch 1970-01-01T00:00:00.000 . Alternatives: As above but don't support the first two granularities ('Year', 'Year and month') on the grounds that they are really matching xs:gYear and xs:gYearMonth. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

DFDL WG action 153 : dfdl:calendarTimezone
by Steve Hanson 13 Jan '12

13 Jan '12

Minutes from DFDL WG call from last year: 13.11. The DFDL property combination of dfdl:calendarTimezone and dfdl:calendarObserveDST is insufficient as it does not cater for different start dates for DST. To achieve this we would have to relax dfdl:calendarTimeZone rules and allow it to take other formats, such as: a) Olson format (eg, "Europe/Paris") b) Unicode short time zone IDs as per (Unicode Locale Data Markup Language ) spec (eg, "frpar") The list of TZs for b) is here: http://lawiki.org/extensions/core/common/bcp47/timezone.xml Note there a special value for 'unknown' - "unk" or "Etc/Unknown" - prefer to empty string? Agreed in principle, raised action 153 to decide on the extended format to use. Noted that calendarObserveDST would still be honoured.for UTC values. Action 153 proposal Allow dfdl:calendarTimezone to be either UTC offset format or Olson format, as per option (a) above. Olson is the most widely-used naming convention for time zones. Option (b) above refers to 5 character UN LOCODE codes (where possible) which identify locations and map to Olson time zones. However they are not well known and therefore less usable. For representing 'not specified' time zone stick with the current errata (2.50) and use empty string. It seems more obvious than "Etc/Unknown" which is not defined in the Olson database, so would have to be handled as a special case. When unparsing and dfdl:calendarPattern contains a formatting symbol for time zone (zzz, Z, VVVV etc) and the infoset value does not contain a time zone, it is a processing error. This matches the behaviour on parsing when the data does not contain a time zone but the pattern does. Spec should be clear on this. When dfdl:calendarObserveDST is set to 'yes', it only makes sense if dfdl: calendarTimeZone is an Olson time zone. If it is a UTC offset then it should be a schema definition error if dfdl:calendarObserveDST is 'yes'. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Updated DFDL 1.0 Errata document
by Steve Hanson 11 Jan '12

11 Jan '12

The errata document is now at v7 and reflects all agreed errata up until 10th January 2012. https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Rescheduled: OGF DFDL Working Group weekly call (17 Jan 16:00 GMT in IBM Hursley DE3S09)
by Steve Hanson 11 Jan '12

11 Jan '12

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl GridForge DFDL: http://forge.ogf.org/projects/dfdl-wg/ Moving one hour later as minuted

1 0

OGF DFDL WG Call Minutes 2012-01-10
by Steve Hanson 10 Jan '12

10 Jan '12

Please find minutes from the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Regards Steve Hanson Architect, DFDL, IBM SWG Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Re: [DFDL-WG] Fw: Inconsistent naming for text bidi properties
by Steve Hanson 10 Jan '12

10 Jan '12

Agreed on WG call to take errata so that second occurrence of 'text' in a bidi property name is removed. That leaves: - textBidiOrdering - textBidiOrientation - textBidiSymmetric - textBidiShaped - textBidiNumeralShape Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org Date: 09/01/2012 14:44 Subject: Fw: Inconsistent naming for text bidi properties To be discussed on this week's DFDL WG call. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 09/01/2012 14:43 ----- From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org Date: 15/12/2011 15:37 Subject: Inconsistent naming for text bidi properties I'd be interested in folks thoughts about this one. If textBidi is 'yes' then the following properties come in to play: - textBidiTextOrdering - textBidiOrientation - textBidiSymmetric - textBidiTextShaped - textBidiNumeralShape Two of the properties have 'text' appearing twice in the name. This seems superfluous to me. Any reason why these two properties have an extra 'text' ? I only noticed this because the IBM DFDL model has coded 'textBidiTextOrientation' which does not match the spec. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

spec clarification needed: is dfdl:terminator='{ ...returns empty string ... }' allowed?
by Mike Beckerle 10 Jan '12

10 Jan '12

Let's use the example of terminator as a delimiter. If I provide an expression so that I can compute terminator at runtime, is it allowed to return empty string? I.e., equivalent to writing dfdl:terminator="" which is effectively "turning off" use of terminator? It seems very problematic to me if we allow this. Nor do I think this generality is needed. We should clarify that for initiator/terminator/separator, if a runtime expression is used, then it must return at least one non-zero-length value. So using a runtime expression for a delimiter is effectively saying "yes there will be a delimiter", you are just not binding its specific value. I believe this runtime expression capability for delimiters was intended to allow the choice of the specific delimiter to be made based on data containing the value. This is common practice in data formats. However, turning on/off whether delimiters are present or not, is not something I anticipated, and it has far bigger implications for the format. I mean you really can't decide much about the data format statically if even the existence of delimiters as part of the format or not can be postponed to runtime. Comments? -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

2 1

OGF DFDL WG Call Agenda 2012-01-10
by Steve Hanson 09 Jan '12

09 Jan '12

Happy New Year to all. Please find agenda attached for the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Fw: Inconsistent naming for text bidi properties
by Steve Hanson 09 Jan '12

09 Jan '12

To be discussed on this week's DFDL WG call. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 09/01/2012 14:43 ----- From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org Date: 15/12/2011 15:37 Subject: Inconsistent naming for text bidi properties I'd be interested in folks thoughts about this one. If textBidi is 'yes' then the following properties come in to play: - textBidiTextOrdering - textBidiOrientation - textBidiSymmetric - textBidiTextShaped - textBidiNumeralShape Two of the properties have 'text' appearing twice in the name. This seems superfluous to me. Any reason why these two properties have an extra 'text' ? I only noticed this because the IBM DFDL model has coded 'textBidiTextOrientation' which does not match the spec. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0