Action 259 was raised last call to decide what to do about the following, as minuted:

Steve has an example of an escape block where the escape block end is not at the end of the un-trimmed data. This gives a processing error. Another IBM product accepts this usage. Should DFDL allow this? Or should there be a new escapeKind that allows escapeBlockStart/End anywhere?

Tried importing these values from a CSV file into an Excel spreadsheet, a Symphony spreadsheet (ie, successor to 123), and also accessing them via ODBC using a Microsoft driver, to compare with IBM DFDL and IBM Cast Iron behaviour.

Test	Data	IBM DFDL	IBM Cast Iron	MS Excel	Lotus Symphony	ODBC
1	This is normal	This is normal	This is normal	This is normal	This is normal	This is normal
2	"This is OK"	This is OK	This is OK	This is OK	This is OK	This is OK
3	"This| is expected"	This| is expected	This| is expected	This| is expected	This| is expected	This| is expected
4	This too "is OK"	This too "is OK"	This too "is OK"	This too "is OK"	This too "is OK"	This too
5	Even "this" is OK	Even "this" is OK	Even "this" is OK	Even "this" is OK	Even "this" is OK	Even
6	"This" is NOT OK	PARSE FAILED	This is NOT OK	This is NOT OK	This is NOT OK	This
7	"This"" is still OK"	This" is still OK	This" is still OK	This" is still OK	This" is still OK	This" is still OK

The data under discussion is 6. It looks like DFDL is out of step with the behaviour of Excel / Symphony spreadsheets, and Cast Iron has adopted that behaviour too.

Out of interest I also checked the output behaviour from Excel. That escaped all instances of embedded quotes in the same way as DFDL, so no issues there.

Regards
 
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU