- dfdl-wg - lists.ogf.org

clarification needed: datetime and time
by Mike Beckerle 29 Jan '13

29 Jan '13

In the below. Should xs:datetime have trailing timezone as well? Right now it has no tz component. Seems to me it should to match iso 8601 format which is what XSD uses. Also, I believe we now think ZZZ is not the right behavior, and seems to be deprecated in ICU. The right thing is ZZZZZ. calendarPatternKind Enum Valid values ‘explicit’, ‘implicit’ ‘explicit’ means the pattern is given by dfdl:calendarPattern, ‘implicit’ means the pattern is derived from the XML schema date/time type. *Logical Type * *Default Pattern* xs:date yyyy-MM-dd xs:dateTime yyyy-MM-dd'T'HH:mm:ss xs:time HH:mm:ssZZZ Annotation: dfdl:element, dfdl:simpleType -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com

3 4

OGF DFDL WG Call Agenda 2013-01-29
by Steve Hanson 29 Jan '13

29 Jan '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/12993?download= Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

representation properties vs. format properties
by Mike Beckerle 29 Jan '13

29 Jan '13

We use both the terms representation property and format property extensively. Do we want to declare these to mean the same thing as each other, or globally replace one with the other. -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com

2 1

Action 193: First draft for errata for RegEx
by Steve Hanson 28 Jan '13

28 Jan '13

Here's a draft errata for action 193, for review on the next WG call. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ============================================================ Section 24 to read as follows: A DFDL regular expression may be specified for the dfdl:lengthPattern format property and the dfdl:testPattern attribute of the dfdl:assert and dfdl:disciminator annotations. DFDL regular expressions do not interpret DFDL entities. A DFDL regular expression is defined by a set of valid pattern characters. For portability, it is recommended that the regular expression pattern is restricted to the inclusive subset of the ICU regular expression [ICURE] and the Java(R) 7 regular expression [JAVARE] with the Unicode character classes flag (UNICODE_CHARACTER_CLASS) turned on. The following regular expression constructs are not common to both ICU and Java(R) 7 and are not recommended in a DFDL regular expression: *Construct* *Meaning* *Notes* \N{UNICODE CHARACTER NAME} Match the named character ICU only \X Match a Grapheme Cluster ICU only \Uhhhhhhhh Match the character with the hex value hhhhhhhh. ICU only (?# ... ) Free-format comment ICU only (?w-w) UREGEX_UWORD - Controls the behaviour of \b in ICU only a pattern. (?d-d) UNIX_LINES - Enables Unix lines mode. Java 7 only (?u-u) UNICODE_CASE - Enables Unicode-aware case folding. Java 7 only - always on for DFDL (?U-U) UNICODE_CHARACTER_CLASS - Enables the Unicode Java 7 only - version of Predefined character classes and POSIX always on for character classes. DFDL (?imsx-imsx:X) X, as a non-capturing group with the given flags. Java 7 only Note that the flags i,s,m,x are valid, but appending :X to the flag is not. Additionally, the behaviour of the word character construct (\w) is not consistent in ICU and Java(R) 7, and is not recommended. In Java (R) 7 \w is [\p{Alpha}\p{gc=Mn}\p{gc=Me}\p{gc=Mc}\p{Digit}\p{gc=Pc}], which is a larger set than ICU where \w is [\p{Ll}\p{Lu}\p{Lt}\p{Lo}\p{Nd}]. Character properties are detailed by the Unicode Regular Expressions [UNICODERE]. Section 30 to add: [ICURE] - http://userguide.icu-project.org/strings/regexp [JAVARE] - http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html [UNICODERE] - http://www.unicode.org/reports/tr18/ Section 30 to remove: [PERLRE] - http://perldoc.perl.org/perlre.html#Extended-Patterns [JAVARE] - http://download.oracle.com/javase/1.4.2/docs/api/java/util/regex/Pattern.ht… Andy Andy Edwards - WebSphere Message Broker - DFDL Email: andy.edwards(a)uk.ibm.com Snail Mail: MP211, Hursley park, Hursley, WINCHESTER, Hants, SO21 2JN Tel int: 247222 Tel ext: +44 (0)1962 817222 Desk: DE2 U20 The Feynman problem solving Algorithm 1) Write down the problem 2) Think real hard 3) Write down the answer -- Murray Gell-mann in the NY Times Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

OGF DFDL WG Call Minutes 2013-01-22
by Steve Hanson 25 Jan '13

25 Jan '13

Please find minutes from the above call on Redmine at http://redmine.ogf.org/dmsf_files/12991?download= Errata document is now at v011 (draft) here: http://redmine.ogf.org/dmsf_files/8104?download= Regards Steve Hanson Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Rescheduled: OGF DFDL WG Extra Call (24 Jan 16:00 GMT in No room)
by Steve Hanson 23 Jan '13

23 Jan '13

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg Meeting was 10am/3pm but only Tim and Steve were on the call

1 0

Clarification when calendarPattern is a superset of the type
by Steve Lawrence 22 Jan '13

22 Jan '13

Assume we have the following element: <xs:element name='date' type='xs:date' dfdl:calendarPattern='yyyy-MM-dd hh:mm' /> So we have a element of type xs:date, but a calendarPattern that specifies date and time formatting symbols. What should happen in this case? Two potential options that I see (there are probably others): 1) Throw some kind of error, signifying that calendarPattern contains formatting symbols not allowed for the type. 2) Truncate the parsed data to fit the type, so the data 2013-01-22 11:59 becomes <date>2013-01-22</date> This seems reasonable, however the spec mentions the following: When unparsing, and property calendarPattern contains a formatting symbol that requires a component of the date/time and the infoset value does not contain that component, it is a processing error. So that means it is impossible to unparse when the calendar pattern is a superset of the element type. Is this ok? Thoughts? - Steve

2 1

Broadcast: OGF DFDL WG Extra Call (23 Jan 15:00 GMT in No room)
by Steve Hanson 22 Jan '13

22 Jan '13

OGF DFDL WG weekly call dial-in details. Passcode for Participants: 5381214 Canada Toll-Free 888-426-6840 China Toll-Free 10-800-711-1071 CHINA NETCOM GROUP USERS China Toll-Free 10-800-110-0996 CHINA TELECOM SOUTH USERS France Toll-Free 0800-94-0558 Germany Toll-Free 0800-000-1018 India Toll-Free 000-800-100-1176 Ireland Toll-Free 1-800-943-427 Israel Toll-Free 1-809-417-783 United Kingdom Caller Paid 0-20-30596451 United Kingdom Toll-Free 0800-368-0638 USA Caller Paid 215-861-6239 USA Toll-Free 888-426-6840 Other international numbers available - e-mail smh(a)uk.ibm.com. OGF DFDL Home: http://www.ogf.org/dfdl Redmine DFDL: http://redmine.ogf.org/projects/dfdl-wg

1 0

Fw: date & time and latest ICU possible issues/conflicts
by Steve Hanson 22 Jan '13

22 Jan '13

Clarification from CLDR via ICU on the single 'V' re-use issue: "V" was originally designed for a variant of pattern "z" in CLDR. It had a valid reason to define a pattern different from "z", but we found that CLDR cannot collect such information differentiating "V" from "z". Also, there were no users who requested us (including ICU) to maintain proper data for differentiating "V" from "z". As the result, the data making the difference between "V" and "z" was deprecated, and we no longer have reason to keep pattern "V" except backward compatibility reason. So, strictly speaking, it's a backward compatibility problem. But - It still produces a text representation of the same calendar field. "V" was not adopted by any other known implementations other than ICU. The distinction between "V" and "z" had never worked as users would expect. "z" is a Java compatible pattern for short abbreviated zone format, and it is available from the very beginning. On the other hand, "V" was a recent addition. CLDR technical committee including myself understand the risk, but all of us preferred cleaner pattern definition over the potential backward compatibility problem. Sound like DFDL should withdraw 'V', I doubt that any IBM customer will be using it. 'VVVV' is ok. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 21/01/2013 18:17 ----- From: Steve Hanson/UK/IBM To: dfdl-wg(a)ogf.org, Date: 17/01/2013 10:04 Subject: Fw: [DFDL-WG] date & time and latest ICU possible issues/conflicts The ICU ticket has been answered, with reference to the following document: http://cldr.unicode.org/development/development-process/design-proposals/ti… 'x'/'X' symbols In a not-too-distant ICU there will be new symbols 'x' and 'X' to handle Z as a time zone for ISO8601 date/times, effectively replacing 'ZZZZZ'. Both 'x' and 'X' will tolerate Z or +00:00 (or variants) on parsing, and on formatting X will result in Z and x will result in +00:00 (or variants). DFDL's use of 'U' is wider than this, as we allow 'U' to appear with any number of 'Z's, meaning that Z is accepted with non-ISO8601 date/times. DFDL also adds the use of 'I' symbol on its own to mean any ISO8601 compliant date/time, and again we allow 'U' to appear with 'I'. However the motivating use case for adding 'U' was IBM MRM which supports this today. But it does so primarily for XML use cases, in particular ISO8601. I am not personally aware of an actual non-XML use case. I suggest we drop the DFDL-specific use of the 'U' symbol in conjunction with 'Z' and 'I' symbols from the DFDL specification via errata, and allow the use of 'ZZZZZ' instead, which at least will accept Z when parsing. When 'x'/'X' support appears in ICU, we can take a future errata to support it or leave until DFDL 2.0. IBM DFDL already supports 'U' but I am ok with deprecating it as I don't believe it will be being used for real. 'V' symbol In a not-too-distant ICU there will be new symbols 'VV' and 'VVV' to handle time zones expressed as Time Zone Ids and localized locations, respectively. We can add that via errata in the future, or leave until DFDL 2.0. However, at the same time the meaning of V is changed slightly. DFDL supports 'V'. I have asked ICU for a clarification. 'O' symbol In a not-too-distant ICU there will be new symbol 'O' to handle localized GMT format variants. We can add that via errata in the future, or leave until DFDL 2.0. Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 17/01/2013 08:58 ----- From: Steve Hanson/UK/IBM To: Mike Beckerle <mbeckerle.dfdl(a)gmail.com>, Cc: dfdl-wg(a)ogf.org Date: 16/01/2013 18:20 Subject: Re: [DFDL-WG] date & time and latest ICU possible issues/conflicts ICU ticket raised as the help does not give an example. https://icu.sanjose.ibm.com/gcoctrac/ticket/469#ticket Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: dfdl-wg(a)ogf.org, Date: 16/01/2013 15:11 Subject: [DFDL-WG] date & time and latest ICU possible issues/conflicts Sent by: dfdl-wg-bounces(a)ogf.org Steve Lawrence is on the Daffodil Open Source DFDL team (on CC), and he has dug into date/time types. He raised some concerns to me that I really haven't been tracking at all, so I wanted to put in front of the rest of the group. The date time format syntax for the latest version of icu4j contains a 'U' character, which means "cyclic year name". However, the daffodil spec says the 'U' character, following a Z makes it so a timezone of UTC is represented as Z instead of +00:00. This seems to be a conflict, and would prevent us from ever upgrading to the newest version of ICU (which might be a good idea). I will point out that the latest version of ICU supports ZZZZZ (5 Z's), which is the ISO8601 timezone format. This doesn't add all the functionality that the DFDL 'U' gives. My question is, is this enough? Are there cases where the ZU, ZZU, etc. are necessary? I'm just concerned that the U is going to quite a bit more complexity, and want to make sure the updates to latest ICU don't address the DFDL-WG concerns. And if we still need the 'U', maybe it should change to a different letter to prevent conflicts with the latest ICU4J? I would point out that the ICU pattern language cannot deal with dual-purpose letters very well, i.e., ambiguities are introduced if the same letter both introduces a format, and if following another format string, modifies its behavior. E.g., does ZU mean Z modified by U, or Z first, and then U. So it seems pretty unfortunate if the ICU libraries added a conflicting use of letter U. I believe the point of DFDL's use of the U modifier for letters I and Z was to be absolutely clear on the GMT timezone 'Z' issue, i.e., to indicate that 'Z' is to be used, and -00:00 is not to be output, nor accepted when parsing. The ICU specification ZZZZZ says ISO format, but that allows either 'Z' or -00:00 to be used for GMT timezone, and it's not clear what it means on output. -- Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2 1

OGF DFDL WG Call Agenda 2013-01-22
by Steve Hanson 21 Jan '13

21 Jan '13

Please find agenda for call on Redmine at http://redmine.ogf.org/dmsf_files/12991?download= Please note that all 2012 calls have been moved to Archive - 2012 folder ( http://redmine.ogf.org/dmsf/dfdl-wg?folder_id=5486) Regards Steve Hanson Architect, IBM Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0