Agreed on call to add in these descriptions,
minus the footnotes.
Errata will be raised to add EEEEEE
and eeeeee.
There are several bugs in ICU, all of
which should ideally be documented in the release notes for a DFDL implementation.
The broken EEEEE behaviour and the ICU4C v ICU4J differences both come
under this.
Regards
Steve Hanson
Architect, IBM Data Format Description Language (DFDL)
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From:
Mike Beckerle <mbeckerle.dfdl@gmail.com>
To:
Steve Hanson/UK/IBM@IBMGB,
Cc:
"dfdl-wg@ogf.org"
<dfdl-wg@ogf.org>
Date:
14/08/2013 14:23
Subject:
Re: [DFDL-WG]
Action 204: Establish strict versus lax behaviour for ICU calendar patterns
This is helpful.
Given where we are, let's just put this in as doc of what
strict and lax mean.
I'm in favor of adding the variations of EEEE... and eeee... which are
supported by ICU. This is upward compatible, and will avoid need for a
special check to exclude them.
The broken EEEEE form is just a bug - I'd say this is
just a release note item for products providing DFDL, unless ICU fixes
it 'real soon now'.
On Wed, Aug 14, 2013 at 8:58 AM, Steve Hanson <smh@uk.ibm.com>
wrote:
For the subset of ICU symbols
that DFDL supports, here is what ICU claim:
1) Lenient parsing behaviour when in 'strict' mode:
a) case insensitive matching for text fields
b) MMM, MMMM, MMMMM all accept either short or long form of Month
c) E, EE, EEE, EEEE, EEEEE
**, EEEEEE
*** all accept either abbreviated,
full, narrow and short forms of Day of Week
d) accept truncated leftmost numeric field (eg, pattern "HHmmss"
allows "123456" (12:34:56) and "23456" (2:34:56) but
not "3456")
2) Additional lenient parsing behaviour when in 'lax' mode:
a) values outside valid ranges are normalized (eg, "March 32 1996"
is treated as "April 1 1996")
b) ignoring a trailing dot after a non-numeric field
c) leading and trailing whitespace in the data but not in the pattern is
accepted ****
d) whitespace in the pattern can be missing in the data
e) partial matching on literal strings (eg, data "20130621d"
allowed for pattern "yyyyMMdd'date' " ****
** Bug found when testing
this - EEEEE 'narrow' form completely broken - ICU ticket raised.
*** EEEEEE and eeeeee
are new and support a 2 char version of 'short' form - eg Tu or Mo. Not
currently allowed by DFDL, we should consider allowing it.
**** Only currently
in ICU4C. ICU4J will be changed to match ICU4C.
Note: IBM is in discussion with
ICU to provide a 'really strict' mode (name tbd) which has no leniency
at all. We need to decide whether to reflect all three variants in the
dfdl:calendarCheckPolicy, or whether to remap our 'strict' to the new 'really
strict' mode when it appears. Given where we are I think is a DFDL 2.0
item.
Regards
Steve Hanson
Architect, IBM Data Format Description Language (DFDL)
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
--
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU