- dfdl-wg - lists.ogf.org

Re: [DFDL-WG] delimited binary data - clarifications
by Steve Hanson 08 Nov '17

08 Nov '17

1. Yes, all delimiters allowed 2. No restriction on the encoding of such delimiters 3. Escape schemes only apply to text representation The dfdl:escapeSchemeRef property only appears in section 13.2 Properties Common to All Simple Types with Text representation. When creating a DFDL schema that involves delimited binary data, you have to be careful that your data can't contain any bytes that match any in-scope delimiter. I believe that IBM DFDL's byte scanner converts in-scope delimiters into the equivalent bytes using the dfdl:encoding of the object, then matches the bytes. Regards Steve Hanson IBM Hybrid Integration, Hursley, UK Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890 From: Mike Beckerle <mbeckerle.dfdl(a)gmail.com> To: "dfdl-wg(a)ogf.org" <dfdl-wg(a)ogf.org>, Josh Adams <jadams(a)tresys.com> Date: 08/11/2017 13:49 Subject: [DFDL-WG] delimited binary data - clarifications Sent by: "dfdl-wg" <dfdl-wg-bounces(a)ogf.org> Daffodil project is implementing various packed formats, and looking at the TLOG schema on the DFDL Schemas site. The DFDL spec is clear that lengthKind delimited is allowed for packed formats (all variants of packed) and hexBinary. My question is whether there is any restriction on the generality of this that was intended, but not stated in the spec, where we should be issuing a clarification. E.g., 1. Can binary data have all of initiators, terminators, and separators? 2. Is there a restriction on the charset encoding used to specify these, e.g., SBCS? Or do the byte patterns being used to scan for these require conversion of the specified delimiter to bytes from any supported encoding? 3. Do escape schemes apply to delimited binary? If, in fact, all these things are allowed, then I believe we should add a one-liner to section 12.3.2.2 specifying that all aspects of delimited parsing including the above, are specifically allowed. -- dfdl-wg mailing list dfdl-wg(a)ogf.org https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ogf.org_mailman_li… Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

delimited binary data - clarifications
by Mike Beckerle 08 Nov '17

08 Nov '17

Daffodil project is implementing various packed formats, and looking at the TLOG schema on the DFDL Schemas site. The DFDL spec is clear that lengthKind delimited is allowed for packed formats (all variants of packed) and hexBinary. My question is whether there is any restriction on the generality of this that was intended, but not stated in the spec, where we should be issuing a clarification. E.g., 1. Can binary data have all of initiators, terminators, and separators? 2. Is there a restriction on the charset encoding used to specify these, e.g., SBCS? Or do the byte patterns being used to scan for these require conversion of the specified delimiter to bytes from any supported encoding? 3. Do escape schemes apply to delimited binary? If, in fact, all these things are allowed, then I believe we should add a one-liner to section 12.3.2.2 specifying that all aspects of delimited parsing including the above, are specifically allowed.

1 0

Cancelled: OGF DFDL Working Group call
by Steve Hanson 31 Oct '17

31 Oct '17

1 0

OGF DFDL WG Call 2017-10-31 - cancelled
by Steve Hanson 31 Oct '17

31 Oct '17

Apologies for late notice - a deadline means I can't make the call today Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Cancelled: OGF DFDL Working Group call
by Steve Hanson 17 Oct '17

17 Oct '17

1 0

Confirmed: OGF DFDL Working Group call (17 Oct 15:00 GDT in IBM: Hursley DE3S09)
by Steve Hanson 17 Oct '17

17 Oct '17

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Agenda for next OGF DFDL WG Call
by Steve Hanson 17 Oct '17

17 Oct '17

Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13567?download= Regards Steve Hanson IBM Hybrid Integration Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890

1 0

Missing fn prefix on count function in 14.5 Hidden Groups example
by Mike Beckerle 12 Oct '17

12 Oct '17

The example shows an expression "{ count(.... ) }". This should be fn:count with the prefix. It would be very odd to bind the default namespace to the XPath function namespace for these tiny expressions. I can imagine someone doing that if they have a large expression, but in this example we should be more explicit. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>

2 1

Latest OGF DFDL WG Call Minutes
by Steve Hanson 04 Oct '17

04 Oct '17

Please find minutes from the latest call at https://redmine.ogf.org/projects/dfdl-wg/news Regards Steve Hanson IBM Hybrid Integration Architect, IBM DFDL, Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890

1 0

OGF DFDL WG Call Agenda 2017-10-03
by Steve Hanson 03 Oct '17

03 Oct '17

Please find agenda for call on Redmine at https://redmine.ogf.org/dmsf_files/13566?download= Regards Steve Hanson IBM Hybrid Integration Architect, IBM DFDL Co-Chair, OGF DFDL Working Group smh(a)uk.ibm.com tel:+44-1962-815848 mob:+44-7717-378890

1 0