- dfdl-wg - lists.ogf.org

DFDL: Minutes from OGF WG call, 20 Feb 2008
by Ian W Parkinson 21 Feb '08

21 Feb '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 20 Feb 2008 Attendees Steve Hanson (IBM) Suman Kalia (IBM) Ian Parkinson (IBM) 1. Discriminators Suman had proposed, via email, that forward references be allowed in discriminator expressions. This would make some DFDL schemas clearer by allowing the discriminator to reside on the element immediately beneath the "choice" construct, rather than on whichever subelement the discrimator uses. However it is simpler to just disallow forward references. Steve and Suman agreed to leave forward references disallowed for the time being, and to see what feedback is received from adopters. The group had previously noted that no timing property is defined on discriminators, as they are for asserts, and Steve raised the question, suggesting that it had perhaps been deliberately omitted when discriminators were introduced as an alternative to allowing an assert to optionally discriminate a choice. Ian observed that if a parse error occurs while parsing an element, then the choice cannot resolve to the affected branch; so we can always evaluate discriminators after parsing the element defining the discriminator. Any parse error would result in a backtrack. We should document the behaviour of discriminators more clearly in the specification. Suman suggested it would be useful to add discriminators automatically, when converting a COBOL copybook, C structure, etc. to DFDL. 2. Abstract data model Steve was hoping that the UML diagrams would allow a DFDL user to quickly and easily see whch subset of XSD is used by DFDL. With this in mind he'd like to fnd a way to simplify the current proposed diagrams, and will think further on this and contact Simon. 3. Entities With his most recent DFDL entities proposal, Alan had asked whether we should support standard XML entities - something which is not listed in the DFDL specification. As entities are defined by XML and not by XML Schema, we are not at liberty to disallow standard XML entities from a DFDL schema. Indeed, there is no obvious reason to disallow these entities in any case. Meeting closed, 17:30 GMT Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: DFDL: Minutes from OGF WG call, 13 Feb 2008
by Suman Kalia 20 Feb '08

20 Feb '08

I have reworked the example where the choice branch condition was specified outside the choice contents. We don't have to specifically mention switch, case statement programming paradigm in DFDL; thinking more about this, I believe switch/case statement paradigm is not relevant here. Attached is the reworked example . I also created another example by importing COBOL copy book (containing redefine clause) using IBM message broker toolkit. Here the branch condition is described inside the choice . In this case the discriminator contain forward reference to check the condition value; implementations may want to restrict the forward reference to the first token within the contents of choice branch. Note : Discriminators are useful when user is dealing with binary or text data which is not tagged.. Your comments/ suggestions are most welcome and appreciated.. Suman Kalia IBM Toronto Lab WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 02/14/2008 02:47 PM ----- Ian W Parkinson <PARKIW(a)uk.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 02/14/2008 01:34 PM To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] DFDL: Minutes from OGF WG call, 13 Feb 2008 Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 13 Feb 2008 Attendees Mike Beckerle (Oco) Steve Hanson (IBM) Suman Kalia (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) Apologies Alan Powell (IBM) 1. Specification Draft Status Alan has distributed draft 31 of the DFDL specification. The meeting reviewed the plan for the next few versions of the drafts and updated it to reflect progress and revised target dates - a copy of the updated plan is attached to these minutes. Simon's UML diagrams describing the schema components are now planned for inclusion in draft 32 but require further discussion. This discussion will be added to the agenda for the DFDL-WG call on 27th February, but Simon would appreciate comments via e-mail before then. These diagrams are intended to set a conceptual model for DFDL and to show where annotations may be attached, but will not be used more formally, e.g. to automatiically generate APIs. The work on nulls/defaults/optionals is complete, except for some small details, which will be included in draft 32. The 'valueCalc' work has been progressed but is not complete, and is also now targeted for draft 32. Other items originally planned for draft 31 are complete and have been included in the draft. 2. Assertions, Discriminators and Choice Suman has distributed an example showing the use of discriminators in choice constructs in DFDL schemas. The meeting discussed the distinction between discriminators and assertions - Mike described an assertion as simply a predicate which, if encountered within a choice, can cause backtracking. In contrast a succesful discriminator expression would lock the choice into a particular branch. If no discriminator matches, then the parse would fail - unless, as Simon pointed out, the choice itself was optional. Simon also suggested that the last branch of a choice could be left without a discriminator to act a a catch-all, but felt that the purpose of a discriminator should be more to help disambiguate between the possible branches rather than form such a "swtich" construct. Steve asked about the timing attribute on assertions, and in particular whether we'd need a similar attribute for discriminators. Mike suggested that the timing attribute might have been included simply to make implementation easier; as without it an implementation would need to perform a significant amount of static analysis. The motivation for discriminators was to allow a choice to be resolved by data encountered before the choice. However as Steve and Suman thought there were use cases where a discriminator might need to refer to elements inside the choice, and so a timing option would be useful. Suman will prepare such an example, and Mike will schedule a further discussion of this topic. Simon distributed an alternative example, showing the use of fixed fields instead of discriminators or assertions., which he felt might form a useful starting point for a full description of choice disambiguation. (Steve left the meeting) 3. Presentation for next OGF conference Simon suggested that Mike highlight the recent discussion topics of the working group, and items which have recently been added to the specification, and asked whether it would be useful to include the UML diagrams. Mike would like to display the diagrams and see whether they trigger a discussion amongst the delegates. He would also like to present work on variable markup and valueCalc. Meeting closed, 18:00 GMT Attachment: revised plan for specification drafts Draft 31: Improve (finish?) nulls/defaults/optionals (Mike, with input from Steve) - Done, apart from minor edit task Expression language (Alan) - Done Property precedence for parsing (Steve) - Done Entities, including basic white space (Alan) - Done Draft 32 ("vX+2"): valueCalc (Mike) - Feb 27, 2008 Remaining aspects of null/default/optionals (Alan) - Mar 5, 2008 2-level description of schema components, including UML (Simon) - Feb 27, 2008 Property precedence for writing (Steve) - Feb 15, 2008 Variable markup (Steve) - Feb 29, 2008 Regular expressions for lengths (Alan) Bring supplements up-to-date (Steve) - Mar 7, 2008 Assertions, discriminators and choice, including discussion of timing option (Suman) - Feb 19, 2008 How speculative parsing works (combining choice and variable-occurence - currently these are separate) (TX person) Reordering the properties discussion: move representation earlier, improve flow of topics (Alan) Draft 33: ("vX+3"): Escape schemes (Ian P) - Mar 21, 2008 String XML type (Ian P) - Mar 21, 2008 Variables (Mike) Selectors (Suman) - Mar 3, 2008 Improvements on property descriptions (All - split TBD) Envelopes and Payloads (Steve) - Mar 5, 2008 Extraneous to spec: Develop Schema for DFDL xsd (Suman) - Mar 15, 2008 Develop Schema for Schema DFDL Subset xsd (Suman) - Mar 30, 2008 (might not be needed) Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

2 3

Discriminator example, alternative presentation
by Simon Parker 15 Feb '08

15 Feb '08

Good evening. As discussed at the conference, here's a more succinct way to express this simple requirement. Perhaps discriminator's motivating example should be more complex. Comments and corrections welcome, Simon Simon Parker Software Consultant PolarLake 80 Harcourt Street Dublin 2, Ireland Phone +353 1 449-1075 Fax +353 1 449-1011 Web http://www.polarlake.com/ The information transmitted in this email is intended for the addressee only and may contain confidential and/or privileged material. Any review, retransmission, dissemination, reliance upon or other use of this information by persons or entities other than the addressee is prohibited. If you think, for any reason, that this message may have been addressed to you in error, we would ask you to notify the sender immediately by return email and delete the material. PolarLake Limited | Registered in Dublin, Ireland | Number 357324 | Registered office as above.

2 2

DFDL property precedence - unparsing
by Steve Hanson 15 Feb '08

15 Feb '08

Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 13 Feb 2008
by Ian W Parkinson 14 Feb '08

14 Feb '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 13 Feb 2008 Attendees Mike Beckerle (Oco) Steve Hanson (IBM) Suman Kalia (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) Apologies Alan Powell (IBM) 1. Specification Draft Status Alan has distributed draft 31 of the DFDL specification. The meeting reviewed the plan for the next few versions of the drafts and updated it to reflect progress and revised target dates - a copy of the updated plan is attached to these minutes. Simon's UML diagrams describing the schema components are now planned for inclusion in draft 32 but require further discussion. This discussion will be added to the agenda for the DFDL-WG call on 27th February, but Simon would appreciate comments via e-mail before then. These diagrams are intended to set a conceptual model for DFDL and to show where annotations may be attached, but will not be used more formally, e.g. to automatiically generate APIs. The work on nulls/defaults/optionals is complete, except for some small details, which will be included in draft 32. The 'valueCalc' work has been progressed but is not complete, and is also now targeted for draft 32. Other items originally planned for draft 31 are complete and have been included in the draft. 2. Assertions, Discriminators and Choice Suman has distributed an example showing the use of discriminators in choice constructs in DFDL schemas. The meeting discussed the distinction between discriminators and assertions - Mike described an assertion as simply a predicate which, if encountered within a choice, can cause backtracking. In contrast a succesful discriminator expression would lock the choice into a particular branch. If no discriminator matches, then the parse would fail - unless, as Simon pointed out, the choice itself was optional. Simon also suggested that the last branch of a choice could be left without a discriminator to act a a catch-all, but felt that the purpose of a discriminator should be more to help disambiguate between the possible branches rather than form such a "swtich" construct. Steve asked about the timing attribute on assertions, and in particular whether we'd need a similar attribute for discriminators. Mike suggested that the timing attribute might have been included simply to make implementation easier; as without it an implementation would need to perform a significant amount of static analysis. The motivation for discriminators was to allow a choice to be resolved by data encountered before the choice. However as Steve and Suman thought there were use cases where a discriminator might need to refer to elements inside the choice, and so a timing option would be useful. Suman will prepare such an example, and Mike will schedule a further discussion of this topic. Simon distributed an alternative example, showing the use of fixed fields instead of discriminators or assertions., which he felt might form a useful starting point for a full description of choice disambiguation. (Steve left the meeting) 3. Presentation for next OGF conference Simon suggested that Mike highlight the recent discussion topics of the working group, and items which have recently been added to the specification, and asked whether it would be useful to include the UML diagrams. Mike would like to display the diagrams and see whether they trigger a discussion amongst the delegates. He would also like to present work on variable markup and valueCalc. Meeting closed, 18:00 GMT Attachment: revised plan for specification drafts Draft 31: Improve (finish?) nulls/defaults/optionals (Mike, with input from Steve) - Done, apart from minor edit task Expression language (Alan) - Done Property precedence for parsing (Steve) - Done Entities, including basic white space (Alan) - Done Draft 32 ("vX+2"): valueCalc (Mike) - Feb 27, 2008 Remaining aspects of null/default/optionals (Alan) - Mar 5, 2008 2-level description of schema components, including UML (Simon) - Feb 27, 2008 Property precedence for writing (Steve) - Feb 15, 2008 Variable markup (Steve) - Feb 29, 2008 Regular expressions for lengths (Alan) Bring supplements up-to-date (Steve) - Mar 7, 2008 Assertions, discriminators and choice, including discussion of timing option (Suman) - Feb 19, 2008 How speculative parsing works (combining choice and variable-occurence - currently these are separate) (TX person) Reordering the properties discussion: move representation earlier, improve flow of topics (Alan) Draft 33: ("vX+3"): Escape schemes (Ian P) - Mar 21, 2008 String XML type (Ian P) - Mar 21, 2008 Variables (Mike) Selectors (Suman) - Mar 3, 2008 Improvements on property descriptions (All - split TBD) Envelopes and Payloads (Steve) - Mar 5, 2008 Extraneous to spec: Develop Schema for DFDL xsd (Suman) - Mar 15, 2008 Develop Schema for Schema DFDL Subset xsd (Suman) - Mar 30, 2008 (might not be needed) Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: Agenda for call 2007-02-13
by Suman Kalia 13 Feb '08

13 Feb '08

Action Item for Vx+2 : - assertions/discriminators and choice (Suman) Attached is an example of how discriminators can be used in DFDL to identify a particular branch within choice.. <?xml version="1.0" encoding="UTF-8"?> <xsd:schema targetNamespace="http://www.example.org/choiceDiscriminators" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dfdl=" http://dataformat.org/" xmlns:tns="http://www.example.org/choiceDiscriminators"> <xsd:element name="multiRecord" type="tns:MultiRecordType"></ xsd:element> <xsd:complexType name="MultiRecordType"> <xsd:sequence> <xsd:element name="recordType" type="xsd:string" ></xsd:element>  <xsd:choice> <xsd:sequence> <xsd:annotation> <xsd:appinfo source=" http://dataformat.org/">  < dfdl:discriminator test="..\recordType='name'" />  </xsd:appinfo> </xsd:annotation> <xsd:element name="firstName" type ="xsd:string" /> <xsd:element name="lastName" type= "xsd:string" /> </xsd:sequence> <xsd:sequence> <xsd:annotation> <xsd:appinfo source=" http://dataformat.org/"> < dfdl:discriminator test="..\recordType='addr'" />  </xsd:appinfo> </xsd:annotation> <xsd:element name="streetAddr" type="xsd:string" /> <xsd:element name="city" type= "xsd:string" /> <xsd:element name="state" type= "xsd:string" /> <xsd:element name="zip" type= "xsd:string" /> </xsd:sequence> <xsd:element name="otherElem" type= "xsd:string"> <xsd:annotation> <xsd:appinfo source=" http://dataformat.org/"> < dfdl:discriminator test="..\recordType='oth" />  </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:choice> </xsd:sequence> </xsd:complexType> </xsd:schema> Suman Kalia IBM Toronto Lab WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 02/12/2008 06:59 PM ----- "Mike Beckerle" <mbeckerle.dfdl(a)gmail.com> Sent by: dfdl-wg-bounces(a)ogf.org 02/12/2008 06:24 PM To <dfdl-wg(a)ogf.org> cc Subject [DFDL-WG] Agenda for call 2007-02-13 Topics for this call include – · Content for OGF22 Meeting. – I need to know what features of our progress we want to highlight. The section about status and progress is where this content goes. Slides 37 through 51 of the OGF21 deck, which is here: http://forge.ogf.org/sf/go/doc15029?nav=1 · Comments on Draft 031 · Review of next tasks Old task list below for reference: Planned (Roughly) vX+1 - 2-level description (schema components) - including UML (Simon) - regular expressions for lengths (Alan) - valueCalc (Mike) - property precedence (Steve) - entities (Alan) - white space handling (Steve) vX+2 - bring supplements up-to-date (Steve) - assertions/discriminators and choice (Suman) - how speculative parsing works (combining choice and variable-occurrence - currently these are separate) (TX person) - reordering the properties discussion (move representation earlier, improve flow of topics) (Alan) vX+3 - escapeSchemes (Ian P) - string xml type (Ian P) - variables (Mike) - selectors (Suman) - improvements on property descriptions (All - split tbd) - envelopes and payloads (Steve) Extraneous to spec - develop Schema for DFDL xsd (Suman) - develop Schema for Schema DFDL Subset xsd (Suman) -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

2 1

Agenda for call 2007-02-13
by Mike Beckerle 12 Feb '08

12 Feb '08

Topics for this call include - . Content for OGF22 Meeting. - I need to know what features of our progress we want to highlight. The section about status and progress is where this content goes. Slides 37 through 51 of the OGF21 deck, which is here: http://forge.ogf.org/sf/go/doc15029?nav=1 . Comments on Draft 031 . Review of next tasks Old task list below for reference: Planned (Roughly) vX+1 - 2-level description (schema components) - including UML (Simon) - regular expressions for lengths (Alan) - valueCalc (Mike) - property precedence (Steve) - entities (Alan) - white space handling (Steve) vX+2 - bring supplements up-to-date (Steve) - assertions/discriminators and choice (Suman) - how speculative parsing works (combining choice and variable-occurrence - currently these are separate) (TX person) - reordering the properties discussion (move representation earlier, improve flow of topics) (Alan) vX+3 - escapeSchemes (Ian P) - string xml type (Ian P) - variables (Mike) - selectors (Suman) - improvements on property descriptions (All - split tbd) - envelopes and payloads (Steve) Extraneous to spec - develop Schema for DFDL xsd (Suman) - develop Schema for Schema DFDL Subset xsd (Suman)

1 0

ogf-dfdl-v1.0-Core-031.doc
by Steve Hanson 08 Feb '08

08 Feb '08

Draft 031 of spec is posted to GridForge here http://forge.gridforum.org/sf/go/doc15032?nav=1 Content as below: Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 08/02/2008 14:26 ----- Alan Powell/UK/IBM 07/02/2008 17:19 To Steve Hanson/UK/IBM@IBMGB cc Geoff Judd/UK/IBM@IBMGB Subject ogf-dfdl-v1.0-Core-031.doc Steve The following updates have been made in this version Rewrite expression language section Update calculated values Added entities instead of hex escape Added parsing property precedence Added most of the clarifications around defaults, nulls and empty strings I was unable to add the DFDL UML diagrams due to lack of time. These will be done in the next version. Thanks Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 06 Feb 2008
by Ian W Parkinson 07 Feb '08

07 Feb '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 06 Feb 2008 Attendees Steve Hanson (IBM) Geoff Judd (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) Alan Powell (IBM) Apologies Mike Beckerle (Oco) Suman Kalia (IBM) 1. Review of the draft DFDL Schema Abstract Model Simon has distributed, via the DFDL working group mailing list, a draft of the UML description of the DFDL schema components, with diagrams built from scratch but using concepts from a number of different, public, UML descriptions of XML schema. The meeting reviewed and discussed this draft. 'Particle' diagram Steve noted that the diagram has changed some of the object names. Simon agreed that he expected some of these to revert to standard XML nomenclature - but that some concepts don't have an existing naming convention. The diagram allows for "all" groups - which are not supported in DFDL. This is as specified in section 5.1 of the present DFDL specification draft. The diagram shows "Group" as a subclass of "GroupDefinition". The XML schema specification does appear to allow this, but the meeting agreed this should be removed from the diagram. Steve and Geoff suggested that GlobalElement should be modelled as a subclass of ElementDeclaration. 'Type' diagram The diagram distinguishes between GlobalType and TypeDefinition, to allow GlobalTypes to be named while TypeDefinitions remain anonymous. The 'base' relationship should be between TypeReference and SimpleType, not between TypeReference and TypeDefinition. Complex type inheritence is not allowed by DFDL. After consulting the schema specification, Steve withdrew his suggestion that ComplexType should contain a ModelGroup rather than a Particle. Simon pointed out that XML Schema 1.1 will allow particles to take annotations, which may prove useful for DFDL. 'DFDL Annotations' diagram Hidden elements are wrapped inside DFDL annotations. This allows for a DFDL schema to be mapped to an XSD schema by stripping all DFDL annotations; the resulting XSD schema describing a DFDL InfoSet for the DFDL schema. This is a goal of DFDL. Alan asked about the purpose of these digrams. They are intended to model and communicate the valid subset of XML schema, and also show where DFDL annotations can be placed in a DFDL schema. We could also use these diagrams to model which XSD objects each DFDL property affects, as well as showing where the properties may be defined - however Simon felt that as a property is not itself a DFDL annotation, this information is better documented elsewhere. Steve felt that the diagrams included in this draft are overcomplicated, and wondered if we could reduce the number of boxes in the diagrams, for example by removing the distinction between named and anonymous objects. Simon observed that this is an important distinction for DFDL as, for example, named types may be reused in different contexts, and this distinction allows us to describe how properties apply differently to named and anonymous types. With this in mind, the diagrams are as simple as possible. 2. Other business Alan will incorporate the UML diagrams, along with the other planned specification updates, into a new draft of the specification to be distributed on the 7th Feb. Meeting closed, 18:00 GMT Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Re: [DFDL-WG] Action 14: Propose DFDL entity scheme v5
by Alan Powell 07 Feb '08

07 Feb '08

All Latest proposal incorporating comments One question: should DFDL support standard XML entities? I have always assumed so but it is not listed in supported XML schema functions. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Ian W Parkinson/UK/IBM 22/01/2008 15:29 To Alan Powell/UK/IBM@IBMGB cc DFDL-Technical-Core@IBMGB, dfdl-wg(a)ogf.org, "Mike Beckerle" <mbeckerle(a)OCO-INC.COM> Subject Re: [DFDL-WG] Action 14: Propose DFDL entity scheme Hi guys, For reference, and from wikipedia (search "newline"): "The Unicode standard addresses the problem by defining a large number of characters that conforming applications should recognize as line terminators: LF: Line Feed, U+000A CR: Carriage Return, U+000D CR+LF: CR followed by LF, U+000D followed by U+000A NEL: Next Line, U+0085 FF: Form Feed, U+000C LS: Line Separator, U+2028 PS: Paragraph Separator, U+2029" ... so I guess, during parse, any of these sequences should match %NL; (maybe excluding FF and PS as being more significant than a single new line?). I agree with Mike, for unparse we'd presumably need a new property to specify this. Again, from wikipedia, this time regarding whitespace: "In Unicode (Unicode Character Database) the following codepoints are defined as whitespace: U0009-U000D (Control characters, containing TAB, CR and LF) U0020 SPACE U0085 NEL U00A0 NBSP U1680 OGHAM SPACE MARK U180E MONGOLIAN VOWEL SEPARATOR U2000-U200A (different sorts of spaces) U2028 LSP U2029 PSP U202F NARROW NBSP U205F MEDIUM MATHEMATICAL SPACE U3000 IDEOGRAPHIC SPACE" ....so presumably &WSP; would match any of these characters on parse. What should it generate on unparse? Cheers, Ian Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK From: Alan Powell/UK/IBM@IBMGB To: "Mike Beckerle" <mbeckerle(a)OCO-INC.COM> Cc: dfdl-wg(a)ogf.org, DFDL-Technical-Core%IBMGB(a)uk.ibm.com Date: 22/01/2008 14:41 Subject: Re: [DFDL-WG] Action 14: Propose DFDL entity scheme Hi Mike %NL; is a single character <LF> on those target platforms where that is the convention or <CR><LF> on others, etc. This is intended to make it easier for the same dfdl schema to parse messages from different platforms. I know we avoided target platform in DFDL and was expecting that this would cause some debate. This will be a good discussion for tomorrow's call Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 "Mike Beckerle" <mbeckerle(a)OCO-INC.COM> 22/01/2008 13:29 To Alan Powell/UK/IBM@IBMGB, <DFDL-Technical-Core%IBMGB(a)uk.ibm.com>, <dfdl-wg(a)ogf.org> cc Subject RE: [DFDL-WG] Action 14: Propose DFDL entity scheme Is the &NL; supposed to represent a single character? Or can it be a CRLF? There?s no notion of ?the target platform? in DFDL. We?ve specifically avoided this notion on purpose. So we need a separate property like newline=?&CR;&LF;? or newline=?&LF;? if we want &NL; to be meaningful, unless some other property is suitable. There are some other Unicode whitespace and Unicode line-ending characters. Do we want to include those in the definitions of WSP and NL ? I recall there are 4 line-endings in Unicode. ?mikeb From: dfdl-wg-bounces(a)ogf.org [mailto:dfdl-wg-bounces@ogf.org] On Behalf Of Alan Powell Sent: Tuesday, January 22, 2008 6:32 AM To: DFDL-Technical-Core%IBMGB(a)uk.ibm.com; dfdl-wg(a)ogf.org Subject: Re: [DFDL-WG] Action 14: Propose DFDL entity scheme All Attached is the latest proposal for DFDL 'entities' The main changes are: - No longer using XML entities as this proved to not meet all the requirements - New generic mnemonics for <NL> and others to represent the NL on the target platform. Alan Powell MP 211, IBM UK Labs, Hursley, Winchester, SO21 2JN, England Notes Id: Alan Powell/UK/IBM email: alan_powell(a)uk.ibm.com Tel: +44 (0)1962 815073 Fax: +44 (0)1962 816898 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0