When testing I found that the data was
corrupted when I got to > 9 'S' symbols, due to ICU's use of int32 to
store the value. I raised a ticket and it has been accepted as a defect.
But it shows that normal use does not go beyond 9.
Regards
Steve Hanson
Architect, IBM
DFDL
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From:
Mike Beckerle <mbeckerle.dfdl@gmail.com>
To:
Steve Hanson/UK/IBM@IBMGB
Cc:
Andrew Edwards/UK/IBM@IBMGB,
DFDL-WG <dfdl-wg@ogf.org>
Date:
25/08/2015 14:16
Subject:
Re: [DFDL-WG]
Action 284 - agenda item on ICU 'S' symbol
Do we really want to allow "any number of S"
?
Quantum mechanics based on plank's constant and the speed
of light, the smallest unit of time is about 3.3x10-44 seconds, so there's
never going to be a need for more than 45 S's in this universe, at least
until time-travel is discovered. (http://www.physlink.com/Education/AskExperts/ae598.cfm)
This is a place where an "implementation specific
maximum" makes sense to me, though I'd be happy to put a floor under
it like not less than 6 "S".
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology
| www.tresys.com
Please note: Contributions to the DFDL Workgroup's email
discussions are subject to the OGF
Intellectual Property Policy
On Tue, Aug 25, 2015 at 6:54 AM, Steve Hanson <smh@uk.ibm.com>
wrote:
Section 13.11.1 to be updated as follows:
| S
| fractional
second (see note 1)
| Number
| S
SS
SSS
| 2
23
235 |
The count of pattern letters determines the format as indicated in the
table. <---- moved
earlier
When numeric fields abut one another directly, with no intervening delimiter
characters, they constitute a run of abutting numeric fields. Such runs
are parsed specially as described at [ICUDateTime].
Unlike other fields, fractional seconds "S" are padded on the
right with zero. <---
moved earlier
Any number of "S"
symbols may by specified in the pattern.
Implementations must accept
any number of "S" symbols and must support at least millisecond
accuracy. When
the number of "S" symbols exceeds the supported accuracy, excess
fractional seconds are truncated
from the right (not rounded) when parsing, and zeros are added to the right
when unparsing. For example, for xs:time with dfdl:calendarPattern "ss.SSSS"
and millisecond accuracy, parsing data "12.3456" creates infoset
value "00:00:12:345", which when unparsing creates data "12.3450".
Regards
Steve Hanson
Architect, IBM
DFDL
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From: Steve
Hanson/UK/IBM
To: Andrew
Edwards/UK/IBM@IBMGB
Cc: DFDL-WG
<dfdl-wg@ogf.org>
Date: 12/08/2015
09:06
Subject: Re:
[DFDL-WG] OGF DFDL WG Call Agenda 2015-08-11 - agenda item on ICU 'S' symbol
Andy - yes that is the behaviour I am seeing.
Regards
Steve Hanson
Architect, IBM
DFDL
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From: Andrew
Edwards/UK/IBM
To: Steve
Hanson/UK/IBM@IBMGB
Cc: DFDL-WG
<dfdl-wg@ogf.org>
Date: 11/08/2015
17:28
Subject: Re:
[DFDL-WG] OGF DFDL WG Call Agenda 2015-08-11 - agenda item on ICU 'S' symbol
Following on from today's call, the relevant piece of documentation is
in http://icu-project.org/apiref/icu4c/classicu_1_1SimpleDateFormat.html
When numeric fields abut one another directly, with no intervening delimiter
characters, they constitute a run of abutting numeric fields. Such runs
are parsed specially. For example, the format "HHmmss" parses
the input text "123456" to 12:34:56, parses the input text "12345"
to 1:23:45, and fails to parse "1234". In other words, the leftmost
field of the run is flexible, while the others keep a fixed width. If the
parse fails anywhere in the run, then the leftmost field is shortened by
one character, and the entire run is parsed again. This is repeated until
either the parse succeeds or the leftmost field is one character in length.
If the parse still fails at that point, the parse of the run fails.
So it seems that when the 'S' is next to other numeric units in the pattern,
it will be subject to the above behaviour. Therefore:
- A pattern of HHmmssSSS with the input 112233123 will become 11:22:33.123
but the input 1122331234 will trigger an error.
- If the pattern includes a '.' to become HHmmss.SSS, I think the
input 1122331234 will become 11:22:33.123 but I'll try and confirm.
Steve - Does that description match what you were seeing?
HTH,
Andy
| Andy
Edwards - IBM
Integration Bus -
DFDL
|
| Email:
| andy.edwards@uk.ibm.com
|
| Snail
Mail:
| MP211,
Hursley park, Hursley, WINCHESTER, Hants, SO21 2JN
|
| Tel
int:
| 247222
|
| Tel
ext:
| +44
(0)1962 817222
|
| Desk:
| DE3
V17 |
| The
Feynman problem solving Algorithm
1) Write down the problem
2) Think real hard
3) Write down the answer
-- Murray Gell-mann in the NY Times |
From: Steve
Hanson/UK/IBM
To: Andrew
Edwards/UK/IBM@IBMGB
Cc: DFDL-WG
<dfdl-wg@ogf.org>
Date: 11/08/2015
12:53
Subject: Re:
[DFDL-WG] OGF DFDL WG Call Agenda 2015-08-11 - agenda item on ICU 'S' symbol
Hi Andy
Your internal ticket #630 gave rise to external ticket http://bugs.icu-project.org/trac/ticket/10962,
which claims to have fixed the API docs to clarify the behaviour.
| S
| fractional
second - truncates (like other time fields)
to the count of letters when formatting. Appends
zeros if more than 3 letters specified. Truncates at
three significant digits when parsing.
| S
SS
SSS
SSSS
| 2
23
235
2350 |
I can't see anywhere that addresses your point about about abutting versus
non-abutting numeric symbols though?
As far as DFDL spec is concerned, this is what we say today:
| S
| fractional
second (see note 1)
| Number
| S
SS
SSS
| 2
24
235 |
There is no 'note 1', I think the note was made into a normal paragraph,
which reads:
Any number of fractional seconds "S" may by specified in the
pattern and accepted by implementations, but an implementation is free
to represent a limited number of fractional seconds internally. Excess
fractional seconds are truncated, not rounded up. At least millisecond
accuracy must be implemented. Unlike other fields, fractional seconds are
padded on the right with zero.
Regards
Steve Hanson
Architect, IBM
DFDL
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From: Andrew
Edwards/UK/IBM
To: Steve
Hanson/UK/IBM@IBMGB
Date: 11/08/2015
11:56
Subject: Re:
[DFDL-WG] OGF DFDL WG Call Agenda 2015-08-11
Hi Steve
Re agenda item 2 and calendar patterns with 'S', this ICU ticket from last
year might be relevant - https://icu.sanjose.ibm.com/gcoctrac/ticket/630.
It seems that the error reporting may also depend on whether the
pattern has 'S' on it's own or next to other numeric pattern entities.
i.e. 'HHmmssS' is subject to length checking, but 'HHmmss S' is not,
due to the space before the 'S'.
HTH,
Andy
| Andy
Edwards - IBM
Integration Bus -
DFDL
|
| Email:
| andy.edwards@uk.ibm.com
|
| Snail
Mail:
| MP211,
Hursley park, Hursley, WINCHESTER, Hants, SO21 2JN
|
| Tel
int:
| 247222
|
| Tel
ext:
| +44
(0)1962 817222
|
| Desk:
| DE3
V17 |
| The
Feynman problem solving Algorithm
1) Write down the problem
2) Think real hard
3) Write down the answer
-- Murray Gell-mann in the NY Times |
From: Steve
Hanson/UK/IBM@IBMGB
To: dfdl-wg@ogf.org
Cc: Mike
Beckerle <mbeckerle@tresys.com>,
jorge.marizan@gmail.com
Date: 10/08/2015
18:29
Subject: [DFDL-WG]
OGF DFDL WG Call Agenda 2015-08-11
Sent by: dfdl-wg-bounces@ogf.org
Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13489?download=
Regards
Steve Hanson
Architect, IBM Data Format Description Language (DFDL)
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU