- dfdl-wg - lists.ogf.org

DFDL Schema abstract data model 4
by Simon Parker 27 Feb '08

27 Feb '08

Good morning. As discussed yesterday, here is a fresh version with improvements and corrections applied. I added a note to the Discussion section, reproduced here for convenience. --- Yesterday's conference expressed the view that the version 3 model contained more detail than was useful. Version 4 removes some of the new abstractions to bring it closer to version 2, but retains most of the relationships. The new naming conventions (Global*, *Reference, *Definition) didn't find favour, and this version uses established names for abstractions discussed in the XML Schema specification. Other changes correct errors and misunderstandings: only simple types may have explicit base types; DFDL doesn't support All groups or explicit Group constructs in groups; some associations were incorrectly described as aggregations. Dfdl annotations can be used at any annotation point. There are some restrictions, but this model cannot show details clearly so they are left for a different form of presentation or for later addition. --- Regards, Simon Simon Parker Software Consultant PolarLake 80 Harcourt Street Dublin 2, Ireland Phone +353 1 449-1075 Fax +353 1 449-1011 Web http://www.polarlake.com/ The information transmitted in this email is intended for the addressee only and may contain confidential and/or privileged material. Any review, retransmission, dissemination, reliance upon or other use of this information by persons or entities other than the addressee is prohibited. If you think, for any reason, that this message may have been addressed to you in error, we would ask you to notify the sender immediately by return email and delete the material. PolarLake Limited | Registered in Dublin, Ireland | Number 357324 | Registered office as above.

2 1

DFDL WG Weekly call 2008-02-27 Agenda
by Mike Beckerle 27 Feb '08

27 Feb '08

(Note: Mike Beckerle cannot attend today's call.) Notes from the missing: Summary of OGF22 presentation - There is not much to say here. There were about 15 attendees. A few questions, and people were pleased to see progress. No volunteers to implement/review/help out. Next OGF23 is Barcelona, after that OGF24 is Singapore. Those are both this year. Value Calc - example was circulated as powerpoint. Suggest review of this for next call on March 5. Agenda topics scheduled for today include: - Discussion of Simon's UML diagrams for the schema components

1 0

DFDL: test email
by Mike Beckerle 25 Feb '08

25 Feb '08

Sorry to trouble you, but could active DFDL WG members please send an email back to me directly to mbeckerle.dfdl(a)ogf.org. Please let me know if you received an email from me yesterday which contains an attachment of slides about the Computed Values. I did not receive this mail myself from dfdl-wg(a)ogf.org, but I did see someone's vacation responder .... so I am wondering if it was delivered properly. ...mike beckerle

1 1

AUTO: Brian Petrini is on vacation(returning 03/03/2008)
by Brian Petrini 23 Feb '08

23 Feb '08

I am out of the office until 03/03/2008.. Hello, I'm on vacation until Monday March 3. For anything urgent, please contact my manager Sam Kaipa (kaipa(a)us.ibm.com) I'll respond as soon as I can. Thanks, -Brian Note: This is an automated response to your message dfdl-wg Digest, Vol 18, Issue 13 sent on 2/23/08 13:08:26. This is the only notification you will receive while this person is away.

1 0

DFDL: Minutes from OGF WG call, 20 Feb 2008
by Ian W Parkinson 21 Feb '08

21 Feb '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 20 Feb 2008 Attendees Steve Hanson (IBM) Suman Kalia (IBM) Ian Parkinson (IBM) 1. Discriminators Suman had proposed, via email, that forward references be allowed in discriminator expressions. This would make some DFDL schemas clearer by allowing the discriminator to reside on the element immediately beneath the "choice" construct, rather than on whichever subelement the discrimator uses. However it is simpler to just disallow forward references. Steve and Suman agreed to leave forward references disallowed for the time being, and to see what feedback is received from adopters. The group had previously noted that no timing property is defined on discriminators, as they are for asserts, and Steve raised the question, suggesting that it had perhaps been deliberately omitted when discriminators were introduced as an alternative to allowing an assert to optionally discriminate a choice. Ian observed that if a parse error occurs while parsing an element, then the choice cannot resolve to the affected branch; so we can always evaluate discriminators after parsing the element defining the discriminator. Any parse error would result in a backtrack. We should document the behaviour of discriminators more clearly in the specification. Suman suggested it would be useful to add discriminators automatically, when converting a COBOL copybook, C structure, etc. to DFDL. 2. Abstract data model Steve was hoping that the UML diagrams would allow a DFDL user to quickly and easily see whch subset of XSD is used by DFDL. With this in mind he'd like to fnd a way to simplify the current proposed diagrams, and will think further on this and contact Simon. 3. Entities With his most recent DFDL entities proposal, Alan had asked whether we should support standard XML entities - something which is not listed in the DFDL specification. As entities are defined by XML and not by XML Schema, we are not at liberty to disallow standard XML entities from a DFDL schema. Indeed, there is no obvious reason to disallow these entities in any case. Meeting closed, 17:30 GMT Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: DFDL: Minutes from OGF WG call, 13 Feb 2008
by Suman Kalia 20 Feb '08

20 Feb '08

I have reworked the example where the choice branch condition was specified outside the choice contents. We don't have to specifically mention switch, case statement programming paradigm in DFDL; thinking more about this, I believe switch/case statement paradigm is not relevant here. Attached is the reworked example . I also created another example by importing COBOL copy book (containing redefine clause) using IBM message broker toolkit. Here the branch condition is described inside the choice . In this case the discriminator contain forward reference to check the condition value; implementations may want to restrict the forward reference to the first token within the contents of choice branch. Note : Discriminators are useful when user is dealing with binary or text data which is not tagged.. Your comments/ suggestions are most welcome and appreciated.. Suman Kalia IBM Toronto Lab WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 02/14/2008 02:47 PM ----- Ian W Parkinson <PARKIW(a)uk.ibm.com> Sent by: dfdl-wg-bounces(a)ogf.org 02/14/2008 01:34 PM To dfdl-wg(a)ogf.org cc Subject [DFDL-WG] DFDL: Minutes from OGF WG call, 13 Feb 2008 Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 13 Feb 2008 Attendees Mike Beckerle (Oco) Steve Hanson (IBM) Suman Kalia (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) Apologies Alan Powell (IBM) 1. Specification Draft Status Alan has distributed draft 31 of the DFDL specification. The meeting reviewed the plan for the next few versions of the drafts and updated it to reflect progress and revised target dates - a copy of the updated plan is attached to these minutes. Simon's UML diagrams describing the schema components are now planned for inclusion in draft 32 but require further discussion. This discussion will be added to the agenda for the DFDL-WG call on 27th February, but Simon would appreciate comments via e-mail before then. These diagrams are intended to set a conceptual model for DFDL and to show where annotations may be attached, but will not be used more formally, e.g. to automatiically generate APIs. The work on nulls/defaults/optionals is complete, except for some small details, which will be included in draft 32. The 'valueCalc' work has been progressed but is not complete, and is also now targeted for draft 32. Other items originally planned for draft 31 are complete and have been included in the draft. 2. Assertions, Discriminators and Choice Suman has distributed an example showing the use of discriminators in choice constructs in DFDL schemas. The meeting discussed the distinction between discriminators and assertions - Mike described an assertion as simply a predicate which, if encountered within a choice, can cause backtracking. In contrast a succesful discriminator expression would lock the choice into a particular branch. If no discriminator matches, then the parse would fail - unless, as Simon pointed out, the choice itself was optional. Simon also suggested that the last branch of a choice could be left without a discriminator to act a a catch-all, but felt that the purpose of a discriminator should be more to help disambiguate between the possible branches rather than form such a "swtich" construct. Steve asked about the timing attribute on assertions, and in particular whether we'd need a similar attribute for discriminators. Mike suggested that the timing attribute might have been included simply to make implementation easier; as without it an implementation would need to perform a significant amount of static analysis. The motivation for discriminators was to allow a choice to be resolved by data encountered before the choice. However as Steve and Suman thought there were use cases where a discriminator might need to refer to elements inside the choice, and so a timing option would be useful. Suman will prepare such an example, and Mike will schedule a further discussion of this topic. Simon distributed an alternative example, showing the use of fixed fields instead of discriminators or assertions., which he felt might form a useful starting point for a full description of choice disambiguation. (Steve left the meeting) 3. Presentation for next OGF conference Simon suggested that Mike highlight the recent discussion topics of the working group, and items which have recently been added to the specification, and asked whether it would be useful to include the UML diagrams. Mike would like to display the diagrams and see whether they trigger a discussion amongst the delegates. He would also like to present work on variable markup and valueCalc. Meeting closed, 18:00 GMT Attachment: revised plan for specification drafts Draft 31: Improve (finish?) nulls/defaults/optionals (Mike, with input from Steve) - Done, apart from minor edit task Expression language (Alan) - Done Property precedence for parsing (Steve) - Done Entities, including basic white space (Alan) - Done Draft 32 ("vX+2"): valueCalc (Mike) - Feb 27, 2008 Remaining aspects of null/default/optionals (Alan) - Mar 5, 2008 2-level description of schema components, including UML (Simon) - Feb 27, 2008 Property precedence for writing (Steve) - Feb 15, 2008 Variable markup (Steve) - Feb 29, 2008 Regular expressions for lengths (Alan) Bring supplements up-to-date (Steve) - Mar 7, 2008 Assertions, discriminators and choice, including discussion of timing option (Suman) - Feb 19, 2008 How speculative parsing works (combining choice and variable-occurence - currently these are separate) (TX person) Reordering the properties discussion: move representation earlier, improve flow of topics (Alan) Draft 33: ("vX+3"): Escape schemes (Ian P) - Mar 21, 2008 String XML type (Ian P) - Mar 21, 2008 Variables (Mike) Selectors (Suman) - Mar 3, 2008 Improvements on property descriptions (All - split TBD) Envelopes and Payloads (Steve) - Mar 5, 2008 Extraneous to spec: Develop Schema for DFDL xsd (Suman) - Mar 15, 2008 Develop Schema for Schema DFDL Subset xsd (Suman) - Mar 30, 2008 (might not be needed) Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

2 3

Discriminator example, alternative presentation
by Simon Parker 15 Feb '08

15 Feb '08

Good evening. As discussed at the conference, here's a more succinct way to express this simple requirement. Perhaps discriminator's motivating example should be more complex. Comments and corrections welcome, Simon Simon Parker Software Consultant PolarLake 80 Harcourt Street Dublin 2, Ireland Phone +353 1 449-1075 Fax +353 1 449-1011 Web http://www.polarlake.com/ The information transmitted in this email is intended for the addressee only and may contain confidential and/or privileged material. Any review, retransmission, dissemination, reliance upon or other use of this information by persons or entities other than the addressee is prohibited. If you think, for any reason, that this message may have been addressed to you in error, we would ask you to notify the sender immediately by return email and delete the material. PolarLake Limited | Registered in Dublin, Ireland | Number 357324 | Registered office as above.

2 2

DFDL property precedence - unparsing
by Steve Hanson 15 Feb '08

15 Feb '08

Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 13 Feb 2008
by Ian W Parkinson 14 Feb '08

14 Feb '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 13 Feb 2008 Attendees Mike Beckerle (Oco) Steve Hanson (IBM) Suman Kalia (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) Apologies Alan Powell (IBM) 1. Specification Draft Status Alan has distributed draft 31 of the DFDL specification. The meeting reviewed the plan for the next few versions of the drafts and updated it to reflect progress and revised target dates - a copy of the updated plan is attached to these minutes. Simon's UML diagrams describing the schema components are now planned for inclusion in draft 32 but require further discussion. This discussion will be added to the agenda for the DFDL-WG call on 27th February, but Simon would appreciate comments via e-mail before then. These diagrams are intended to set a conceptual model for DFDL and to show where annotations may be attached, but will not be used more formally, e.g. to automatiically generate APIs. The work on nulls/defaults/optionals is complete, except for some small details, which will be included in draft 32. The 'valueCalc' work has been progressed but is not complete, and is also now targeted for draft 32. Other items originally planned for draft 31 are complete and have been included in the draft. 2. Assertions, Discriminators and Choice Suman has distributed an example showing the use of discriminators in choice constructs in DFDL schemas. The meeting discussed the distinction between discriminators and assertions - Mike described an assertion as simply a predicate which, if encountered within a choice, can cause backtracking. In contrast a succesful discriminator expression would lock the choice into a particular branch. If no discriminator matches, then the parse would fail - unless, as Simon pointed out, the choice itself was optional. Simon also suggested that the last branch of a choice could be left without a discriminator to act a a catch-all, but felt that the purpose of a discriminator should be more to help disambiguate between the possible branches rather than form such a "swtich" construct. Steve asked about the timing attribute on assertions, and in particular whether we'd need a similar attribute for discriminators. Mike suggested that the timing attribute might have been included simply to make implementation easier; as without it an implementation would need to perform a significant amount of static analysis. The motivation for discriminators was to allow a choice to be resolved by data encountered before the choice. However as Steve and Suman thought there were use cases where a discriminator might need to refer to elements inside the choice, and so a timing option would be useful. Suman will prepare such an example, and Mike will schedule a further discussion of this topic. Simon distributed an alternative example, showing the use of fixed fields instead of discriminators or assertions., which he felt might form a useful starting point for a full description of choice disambiguation. (Steve left the meeting) 3. Presentation for next OGF conference Simon suggested that Mike highlight the recent discussion topics of the working group, and items which have recently been added to the specification, and asked whether it would be useful to include the UML diagrams. Mike would like to display the diagrams and see whether they trigger a discussion amongst the delegates. He would also like to present work on variable markup and valueCalc. Meeting closed, 18:00 GMT Attachment: revised plan for specification drafts Draft 31: Improve (finish?) nulls/defaults/optionals (Mike, with input from Steve) - Done, apart from minor edit task Expression language (Alan) - Done Property precedence for parsing (Steve) - Done Entities, including basic white space (Alan) - Done Draft 32 ("vX+2"): valueCalc (Mike) - Feb 27, 2008 Remaining aspects of null/default/optionals (Alan) - Mar 5, 2008 2-level description of schema components, including UML (Simon) - Feb 27, 2008 Property precedence for writing (Steve) - Feb 15, 2008 Variable markup (Steve) - Feb 29, 2008 Regular expressions for lengths (Alan) Bring supplements up-to-date (Steve) - Mar 7, 2008 Assertions, discriminators and choice, including discussion of timing option (Suman) - Feb 19, 2008 How speculative parsing works (combining choice and variable-occurence - currently these are separate) (TX person) Reordering the properties discussion: move representation earlier, improve flow of topics (Alan) Draft 33: ("vX+3"): Escape schemes (Ian P) - Mar 21, 2008 String XML type (Ian P) - Mar 21, 2008 Variables (Mike) Selectors (Suman) - Mar 3, 2008 Improvements on property descriptions (All - split TBD) Envelopes and Payloads (Steve) - Mar 5, 2008 Extraneous to spec: Develop Schema for DFDL xsd (Suman) - Mar 15, 2008 Develop Schema for Schema DFDL Subset xsd (Suman) - Mar 30, 2008 (might not be needed) Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: Agenda for call 2007-02-13
by Suman Kalia 13 Feb '08

13 Feb '08

Action Item for Vx+2 : - assertions/discriminators and choice (Suman) Attached is an example of how discriminators can be used in DFDL to identify a particular branch within choice.. <?xml version="1.0" encoding="UTF-8"?> <xsd:schema targetNamespace="http://www.example.org/choiceDiscriminators" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dfdl=" http://dataformat.org/" xmlns:tns="http://www.example.org/choiceDiscriminators"> <xsd:element name="multiRecord" type="tns:MultiRecordType"></ xsd:element> <xsd:complexType name="MultiRecordType"> <xsd:sequence> <xsd:element name="recordType" type="xsd:string" ></xsd:element>  <xsd:choice> <xsd:sequence> <xsd:annotation> <xsd:appinfo source=" http://dataformat.org/">  < dfdl:discriminator test="..\recordType='name'" />  </xsd:appinfo> </xsd:annotation> <xsd:element name="firstName" type ="xsd:string" /> <xsd:element name="lastName" type= "xsd:string" /> </xsd:sequence> <xsd:sequence> <xsd:annotation> <xsd:appinfo source=" http://dataformat.org/"> < dfdl:discriminator test="..\recordType='addr'" />  </xsd:appinfo> </xsd:annotation> <xsd:element name="streetAddr" type="xsd:string" /> <xsd:element name="city" type= "xsd:string" /> <xsd:element name="state" type= "xsd:string" /> <xsd:element name="zip" type= "xsd:string" /> </xsd:sequence> <xsd:element name="otherElem" type= "xsd:string"> <xsd:annotation> <xsd:appinfo source=" http://dataformat.org/"> < dfdl:discriminator test="..\recordType='oth" />  </xsd:appinfo> </xsd:annotation> </xsd:element> </xsd:choice> </xsd:sequence> </xsd:complexType> </xsd:schema> Suman Kalia IBM Toronto Lab WebSphere Business Integration Application Connectivity Tools Tel : 905-413-3923 T/L 969-3923 Fax : 905-413-4850 T/L 969-4850 Internet ID : kalia(a)ca.ibm.com ----- Forwarded by Suman Kalia/Toronto/IBM on 02/12/2008 06:59 PM ----- "Mike Beckerle" <mbeckerle.dfdl(a)gmail.com> Sent by: dfdl-wg-bounces(a)ogf.org 02/12/2008 06:24 PM To <dfdl-wg(a)ogf.org> cc Subject [DFDL-WG] Agenda for call 2007-02-13 Topics for this call include – · Content for OGF22 Meeting. – I need to know what features of our progress we want to highlight. The section about status and progress is where this content goes. Slides 37 through 51 of the OGF21 deck, which is here: http://forge.ogf.org/sf/go/doc15029?nav=1 · Comments on Draft 031 · Review of next tasks Old task list below for reference: Planned (Roughly) vX+1 - 2-level description (schema components) - including UML (Simon) - regular expressions for lengths (Alan) - valueCalc (Mike) - property precedence (Steve) - entities (Alan) - white space handling (Steve) vX+2 - bring supplements up-to-date (Steve) - assertions/discriminators and choice (Suman) - how speculative parsing works (combining choice and variable-occurrence - currently these are separate) (TX person) - reordering the properties discussion (move representation earlier, improve flow of topics) (Alan) vX+3 - escapeSchemes (Ian P) - string xml type (Ian P) - variables (Mike) - selectors (Suman) - improvements on property descriptions (All - split tbd) - envelopes and payloads (Steve) Extraneous to spec - develop Schema for DFDL xsd (Suman) - develop Schema for Schema DFDL Subset xsd (Suman) -- dfdl-wg mailing list dfdl-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/dfdl-wg

2 1