February 2012 - dfdl-wg - lists.ogf.org

DFDL Modeling Question
by Bradley Sexton 02 Mar '12

02 Mar '12

Hello, I've been looking at modeling Rich Text Format (RTF) files using the IBM Message Broker DFDL implementation, and ran into an issue. For some background, here's a small example of an RTF file: {\rtf1\ansi\ansicpg1252\deff0\deflang1033{\fonttbl{\f0\froman\fprq2\fcharset0 Times New Roman;}{\f1\fswiss\fcharset0 Arial;}}{\*\generator Msftedit 5.41.15.1515;}\viewkind4\uc1\pard\f0\fs24 This is an example document of an RTF file.\f1\fs20\par{\*\passwordhash 010000004c000000010000000480000050c300001400000010000000f89c360d0c9d360d000000008bc29e2f78a2144122ed68a1701e2ea50bbbbeaf7333c40dfe048ccf55f709b8cc7e8b49}} '\' and '\*\' mark the beginning of control words, and the curly braces mark the beginning and end of control groups that contain control words and data. My issue is that control words and data do not have suitable terminators for parsing. The end of control words is signified by a space when trailing data is present, but typically they are ended by '\' signalling the beginning of a new word or a curly brace signalling the end of the current of beginning of a new control group. Similarly data is typically ended by the '}' of the parent control group. With the exception of a small header the value and placement of control words, groups, and data varies by file. My issue with modeling this is that I was going to use dfdl:lengthKind="pattern" in lieu of suitable delimiters, but this feature is not implemented by IBM. I'm looking for an alternative way to model the data, and was hoping someone on the mailing list might have suggestions. My goal is to model control words and groups in as general a manner as possible given IBMs implementation restrictions, since RTF has over 1800 defined control words and gives you the ability to create your own. Ideal output for the above sample would be something along these lines: <file> <controlWord>rtf1</controlWord> <controlWord>ansi</controlWord> <controlWord>ansicpg1252</controlWord> <controlWord>deff0</controlWord> <controlWord>deflang1033</controlWord> <controlGroup> <name>fonttbl</name> <controlGroup> <name>f0</name> <controlWord>froman</controlWord> <controlWord>fprq2</controlWord> <controlWord>fcharset0</controlWord> <data>Times New Roman;</data> </controlGroup> <controlGroup> <name>f1</name> <controlWord>fswiss</controlWord> <controlWord>fcharset0</controlWord> <data>Arial;</data> </controlGroup> </controlGroup> <controlGroup> <name>generator</name> <data>Msftedit 5.41.15.1515;</data> </controlGroup> <controlWord>viewkind4</controlWord> <controlWord>uc1</controlWord> <controlWord>pard</controlWord> <controlWord>f0</controlWord> <controlWord>fs24</controlWord> <text>This is an example document of an RTF file.</text> <controlWord>f1</controlWord> <controlWord>fs20</controlWord> <controlWord>par</controlWord> <controlGroup> <name>passwordhash</name> <data>010000004c000000010000000480000050c3. . .</data> </controlGroup> </file> IBM Unsupported Features: http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/index.jsp?topic=%2F… I know that's a lot of info out of left field, but I wanted to try and explain it as thoroughly as possible to avoid any confusion. Thanks in advance for any advice you might have and let me know if I've been unclear in any areas. Bradley Sexton

2 6

OGF DFDL WG Call Minutes 2012-02-28
by Steve Hanson 28 Feb '12

28 Feb '12

Please find minutes from the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Errata document has been updated to version 8 and may be found via https://forge.ogf.org/sf/projects/dfdl-wg Regards Steve Hanson Architect, DFDL, IBM SWG Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

OGF DFDL WG Call Agenda 2012-02-28
by Steve Hanson 27 Feb '12

27 Feb '12

Please find agenda attached for the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0

Spec errata 3.3
by Mike Beckerle 23 Feb '12

23 Feb '12

----- Message from Steve Hanson <smh(a)uk.ibm.com> on Tue, 21 Feb 2012 17:24:59 +0000 ----- To be discussed on next DFDL WG call. As a result of action 136, the spec errata document currently says: 3.3. Section 12.3. Clarify that when property is lengthKind 'explicit', 'implicit', 'prefixed' or 'pattern', it means that delimiter scanning is turned off and in-scope delimiters are not looked for within or between elements. Consequently remove the last paragraph of section 5.2.2 starting "It is a processing error when a fixed-length string is found to have a number of characters not equal to the fixed number". It has been pointed out that for a complex element with lengthKind 'implicit', turning off in-scope delimiters is not a consistent behaviour. Even though implicit means that the length is defined by the child content, this is still subject to constraints imposed by the parent. Further, as lengthKind 'implicit' is the implied lengthKind for local sequences and choices, it should be possible to wrap such a sequence or choice in an element with lengthKind 'implicit' and no framing, and experience no behaviour change in parsing other than the addition to the infoset of the element. The ability to selectively switch off in-scope delimiters is something that could be useful for both 'implicit' and 'delimited' lengthKinds, and if so could be added post 1.0 as a separate control. The proposal is to change errata 3.3 to read: 3.3. Section 12.3. Clarify that when property lengthKind is 'explicit', 'prefixed' or 'pattern', then delimiter scanning is turned off and in-scope delimiters are not looked for within or between elements. Clarify that when property lengthKind is 'implicit' and type is simple, then delimiter scanning is turned off and in-scope delimiters are not looked for within or between elements. Clarify that when property lengthKind is 'implicit' and type is complex, then delimiter scanning is not turned off and in-scope delimiters are looked for. Consequently remove the last paragraph of section 5.2.2 starting "It is a processing error when a fixed-length string is found to have a number of characters not equal to the fixed number". Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ________________________________ Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

OGF DFDL WG Call Minutes 2012-02-21
by Mike Beckerle 23 Feb '12

23 Feb '12

Please find minutes from the above call on GridForge at https://forge.ogf.org/sf/docman/do/downloadDocument/projects.dfdl-wg/docman… Errata document has been updated to version 8 and may be found via https://forge.ogf.org/sf/projects/dfdl-wg Regards Steve Hanson Architect, DFDL, IBM SWG Co-Chair, OGF DFDL Working Group Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 ________________________________ Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Clarification on Escape Schemes being Optional
by Mike Beckerle 21 Feb '12

21 Feb '12

Are escape schemes entirely optional in DFDL v1.0, or just "defineEscapeScheme" named ones? -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

1 0

test message 2
by Mike Beckerle 21 Feb '12

21 Feb '12

Please ignore. -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

1 0

Testing mailing list responsiveness
by Steve Hanson 21 Feb '12

21 Feb '12

Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

test message
by Mike Beckerle 21 Feb '12

21 Feb '12

Please ignore this message. -- Mike Beckerle | OGF DFDL WG Co-Chair Tel: 781-330-0412

1 0

OGF DFDL WG Call Agenda 2012-02-21
by Steve Hanson 21 Feb '12

21 Feb '12

Please find agenda attached for the above call (there is a problem with GridForge so I can't post the document there) Regards Steve Hanson Architect, Data Format Description Language (DFDL) Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh(a)uk.ibm.com tel:+44-1962-815848

1 0