Another related problem:

a, b, notList, c, d

a, b, "list1, list2, list3",c,d

Here the 3rd field is a list, comma separated. Quoted if there is more than one list item.

I think to parse this I have to treat the quotation marks as initiator/terminator, and set dfdl:separator="", but since the quotes are optional for the single-list-item case, I'm going to need a choice.

I think the best I can do is

<ignore:ListOf1__XMLSchemaMakesMeHaveThisForUPA/>

<List>notList</List>

and

<ignore:ListOfN__XMLSchemaMakesMeHaveThisForUPA/>

as the XML representations.

Are there any better/cleaner solutions?

I did think of this way: (note: I've omitted xs:annotation and xs:appinfo for brevity), but it isn't exactly "clean".

This is what I call "modeling syntax as data"....

<dfdl:defineVariable name="foundOpenQuote" type="xs:boolean"/>

<xs:group name="optionalOpenQuote">

<xs:sequence dfdl:initiiator='"'>

<dfdl:setVariable ref="foundOpenQuote" value="{ fn:true() }"/>

</xs:sequence>

<xs:sequence dfdl:initiator=""/>

</choice>

</xs:group>

<xs:group name="matchingCloseQuote">

<xs:sequence dfdl:terminator='"'>

<dfdl:discriminator>{ $foundOpenQuote eq fn:true() }</dfdl:assert>

</xs:sequence>

<xs:sequence />

</choice>

</xs:group>

// The main sequence for the data would then have this as the list element:

<xs:sequence>

<dfdl:newVariableInstance ref="foundOpenQuote" defaultValue="false"/>

<xs:sequence dfdl:hiddenGroupRef="optionalOpenQuote"/>

<xs:sequence dfdl:separator=",">

<xs:element name="List" type="xs:string" maxOccurs="unbounded"/>

</xs:sequence>

<xs:sequence dfdl:hiddenGroupRef="matchingCloseQuote"/>

</xs:sequence>

I'd try this out, except that we haven't got dfdl:newVariableInstance yet.

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com

Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy

On Wed, Nov 22, 2017 at 4:11 AM, Steve Hanson <smh@uk.ibm.com> wrote:

I don't think there is a way to achieve what you want. As you say, trimming pad chars takes precedence over applying escape scheme.

I wondered if you could define the escapeBlockStart and End as "%WSP*; and %WSP*;" respectively but the white space entities are not allowed as escape character or in escape block start/end.

Regards

Steve Hanson
IBM Hybrid Integration, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh@uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890

From: Mike Beckerle <mbeckerle.dfdl@gmail.com>
To: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>
Date: 22/11/2017 01:28
Subject: [DFDL-WG] how to trim inside of escape block?
Sent by: "dfdl-wg" <dfdl-wg-bounces@ogf.org>

I have a CSV file

Some lines look like this

a,b," started with spaces, appearing right after the escape block start ",c,d,e

I reviewed the spec, and I see that pad characters appear outside of the quotation marks (escape block start/end).

What I'm trying to do is remove the whitespace after the escape block start, and before the escape block end. This is just spurious whitespace, appears because some of these CSV files were edited by people.

In my data the quoting characters are not always present. They are only there if a comma appears in the data string.

Is there a technique for getting rid of the leading/trailing whitespace inside the escape block start/end that I have forgotten?

...mikeb

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy
-- dfdl-wg mailing list dfdl-wg@ogf.org https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ogf.org_mailman_listinfo_dfdl-2Dwg&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=AJa9ThEymJXYnOqu84mJuw&m=vfzt-MyHajT591zYQmbcxckPT-mZLjNRPlTrg8kgRgY&s=6PDI_r_U7OUsqAxzv24ZiCuH5zPpWFyzXbneqH1GPXk&e=

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU