WG call 3rd June: DFDL spec will change
so that an escape block end does not have to be the last thing in the data
(after trimming). It must always be present. A new erratum will be raised.
Regards
Steve Hanson
Architect, IBM
DFDL
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From:
Steve Hanson/UK/IBM
To:
dfdl-wg@ogf.org,
Date:
20/05/2014 17:50
Subject:
Re: [DFDL-WG]
Action 259 - Consider allowing more flexible escapeBlock
schemes
As discussed on the call, there is an
import case that is not covered in the table, namely where quotes surround
a delimiter but the opening quote is not at the start of the data. I imported
the following text string into Excel:
This
is "," two separate fields
And indeed two columns were created,
meaning the comma was treated as a delimiter and not escaped. This matches
DFDL so good.
Interestingly, the first column was
as expected...
This
is "
...but the second was not:
two
separate fields
Notice the leading quote was removed
without error, meaning that the absence of the closing quote is permitted!
Regards
Steve Hanson
Architect, IBM
DFDL
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From:
Tim Kimber/UK/IBM@IBMGB
To:
dfdl-wg@ogf.org,
Date:
13/05/2014 15:37
Subject:
Re: [DFDL-WG]
Action 259 - Consider allowing more flexible escapeBlock
schemes
Sent by:
dfdl-wg-bounces@ogf.org
That looks fairly conclusive to me.
DFDL should fall into line with established practice.
regards,
Tim Kimber,
IBM Integration Bus Development (Industry Packs)
Hursley, UK
Internet: kimbert@uk.ibm.com
Tel. 01962-816742
Internal tel. 37246742
From: Steve
Hanson/UK/IBM@IBMGB
To: dfdl-wg@ogf.org,
Date: 13/05/2014
11:50
Subject: [DFDL-WG]
Action 259 - Consider allowing more flexible escapeBlock
schemes
Sent by: dfdl-wg-bounces@ogf.org
Action 259 was raised last call to decide what to do about the following,
as minuted:
Steve has an example of an escape block where the escape block end is not
at the end of the un-trimmed data. This gives a processing error. Another
IBM product accepts this usage. Should DFDL allow this? Or should there
be a new escapeKind that allows escapeBlockStart/End anywhere?
Tried importing these values from a CSV file into an Excel spreadsheet,
a Symphony spreadsheet (ie, successor to 123), and also accessing them
via ODBC using a Microsoft driver, to compare with IBM DFDL and IBM Cast
Iron behaviour.
|
Test
| Data
| IBM
DFDL
| IBM
Cast Iron
| MS
Excel
| Lotus
Symphony
| ODBC
|
|
|
|
|
|
|
|
|
|
1
| This
is normal
| This
is normal
| This
is normal
| This
is normal
| This
is normal
| This
is normal
|
|
2
| "This
is OK"
| This
is OK
| This
is OK
| This
is OK
| This
is OK
| This
is OK
|
|
3
| "This|
is expected"
| This|
is expected
| This|
is expected
| This|
is expected
| This|
is expected
| This|
is expected
|
|
4
| This
too "is OK"
| This
too "is OK"
| This
too "is OK"
| This
too "is OK"
| This
too "is OK"
| This
too
|
|
5
| Even
"this" is OK
| Even
"this" is OK
| Even
"this" is OK
| Even
"this" is OK
| Even
"this" is OK
| Even
|
|
6
| "This"
is NOT OK
| PARSE
FAILED
| This
is NOT OK
| This
is NOT OK
| This
is NOT OK
| This
|
|
7
| "This""
is still OK"
| This"
is still OK
| This"
is still OK
| This"
is still OK
| This"
is still OK
| This"
is still OK |
The data under discussion is 6. It looks like DFDL is out of step with
the behaviour of Excel / Symphony spreadsheets, and Cast Iron has adopted
that behaviour too.
Out of interest I also checked the output behaviour from Excel. That escaped
all instances of embedded quotes in the same way as DFDL, so no issues
there.
Regards
Steve Hanson
Architect, IBM
DFDL
Co-Chair, OGF
DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU--
dfdl-wg mailing list
dfdl-wg@ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU