November 2007 - dfdl-wg - lists.ogf.org

Invitation: DFDL OGF WG - discuss Grammar (Nov 16 09:00 AM EST in Teleconference)
by Mike Beckerle 15 Nov '07

15 Nov '07

Calendar Entry ---------------------------------------------------------------------------------------------------------------------------- Subject Chair DFDL OGF WG - discuss Grammar Mike Beckerle/Worcester/IBM Invitees Date Required (to) 11/16/2007 Time Optional (cc) 09:00 AM - 10:00 AM (1 hour) dfdl-wg(a)ogf.org FYI (bcc) Proposed start -1 at Proposed end -1 at Original start 11/16/2007 at 09:00 AM Original end 11/16/2007 at 10:00 AM Where Location Teleconference I'd like to discuss the DFDL Grammar draft doc before next week, as I am going to miss that meeting also unfortunately. I'm hoping to fold the Infoset material and with luck this grammar stuff back into the core document "real soon now" like Fri 11/16. the grammar note/doc attached again here in case you've lost it. This is in response to the InfoSet work and must be read in context of that work. (See attached file: dfdl-grammar.doc) ----------------------------------------------------------------------------------------------------------------------------------------- Call in details here: Participant Passcode: 605528 ITN: 2-421-0033 Tieline: 421-0033 Participants from countries not listed in the table below should dial the ITN, Tieline or one of the Toll numbers. |------------------+----------------------+--------------------| | Country | Toll-free (IBM Pays) | Toll (Caller Pays) | |------------------+----------------------+--------------------| | Australia | 1-800-24-2283 | | |------------------+----------------------+--------------------| | Austria | 0800-292-738 | +43 179576264 | |------------------+----------------------+--------------------| | Belgium | 0800-7-3026 | +32 22006114 | |------------------+----------------------+--------------------| | Canada | see the USA number | see the USA number | |------------------+----------------------+--------------------| | China North | 10800-711-0770 | | | China South | 10800-110-0713 | | |------------------+----------------------+--------------------| | Czech Republic | 800-143-241 | +420-272-133-001 | | | | or | | | | +420-533-413-001 | |------------------+----------------------+--------------------| | Denmark | 80-888377 | +45 38323070 | |------------------+----------------------+--------------------| | Finland | 0800-914-630 | +358 972519061 | |------------------+----------------------+--------------------| | France | 0800-902366 | +33 157323040 or | | | | +33 157323041 | |------------------+----------------------+--------------------| | Germany | 0800-181-6323 | +49 6951709081 | |------------------+----------------------+--------------------| | Greece | 00-800-11-004-9793 | | |------------------+----------------------+--------------------| | Hungary | 06-800-16-381 | | |------------------+----------------------+--------------------| | India | 000-800-100-1156 | | |------------------+----------------------+--------------------| | Ireland | 1800-558728 | +353 16569209 | |------------------+----------------------+--------------------| | Israel | 1-809-317-098 | +972-3-9165956 | |------------------+----------------------+--------------------| | Italy | 800-788634 | +39 0269430413 | |------------------+----------------------+--------------------| | Mexico | see the USA number | see the USA number | |------------------+----------------------+--------------------| | Netherlands | 0800-022-8558 | +31 20513 4100 | |------------------+----------------------+--------------------| | Norway | 800-18373 | +47 24159528 | |------------------+----------------------+--------------------| | Poland | 0-0800111-1712 or | +48 22 366 5400 or | | | IBM PlusGSM 85400 | +48 22 609 5400 | |------------------+----------------------+--------------------| | Portugal | 800-8-11111 | | |------------------+----------------------+--------------------| | Russia | 8-10-800-2-172-10-44 | | |------------------+----------------------+--------------------| | Slovakia | | +421 2 4954 6539 | | | | or | | | | +421 2 4964-9935 | |------------------+----------------------+--------------------| | South Africa | 0800-981-078 | | |------------------+----------------------+--------------------| | Spain | 900-95-1089 | +34 912754171 | |------------------+----------------------+--------------------| | Sweden | 020-799414 | +46 850163259 | |------------------+----------------------+--------------------| | Switzerland | 0800-564-331 | +41 58 333 7337 | |------------------+----------------------+--------------------| | United Kingdom | 0808-234-1969 | +44 2070260533 | |------------------+----------------------+--------------------| | USA | 877-421-0033 | 770-615-1250 | |------------------+----------------------+--------------------|

1 0

DFDL: Minutes from OGF WG call, 14 Nov 2007
by Ian W Parkinson 15 Nov '07

15 Nov '07

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 14 Nov 2007 Attendees Geoff Judd (IBM) Steve Hanson (IBM) Suman Kalia (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) Apologies Mike Beckerle (IBM) 1. Information Set Proposal Following last week's review of the Information Set proposal, Simon updated the proposal and distributed a new draft, version 7. Alan queried the reference to empty lists under "No Value", asking if this was necessary as DFDL does not use a concept of lists - this is carried over from the XML Information Set specification, and is only theoretical. The proposal will be distributed wider within IBM to solicit final comments. 2. Review of Revised DFDL Data Grammar Mike has updated the Data Grammar portion of the DFDL specification, to take into account the InfoSet proposal, and distributed this via the DFDL-WG mailing list. The meeting reviewed the revised grammar, touching upon the following points: Both the ComplexContent and ComplexElement productions include a "Prefix" element. Alan asked whether this duplication was necessary. Steve and Simon noted that this had not changed during the revision and that there are cases where this is useful. The meeting agreed not to change this. Alan asked why we need both Element and ComplexContent in the SequenceItem production. Element effectively introduces a name. Simon noted that most of the changes were in the productions, and wondered what influence the InfoSet proposal actually had on the specification. Steve said that there had been real changes, for example with SimpleElement and ComplexElement. Steve observed that Prefix and Postfix Separators no longer appear in the productions table. The meeting agreed that these were probably no longer necessary, and that this was a useful simplification, but to check this with Mike. 3. Object Constraint Language The UML diagram in the revised InfoSet proposal uses Object Constraint Language1 (OCL) to specify an invariant on the Element class. Simon asked the meeting to consider whether this might be a useful approach in other sections of the DFDL specification, in particular when describing parser behaviour. Suman was concerned that it might prove useful only for specifying parser internals which should not be covered in the language specification. 4. Other Business Suman asked if we need to define a parser API. Steve suggested that this should not form part of the language specification, but that there may need to be other specification documents. Steve and Alan will be compiling a list of remaining work items to move the specification towards completion. Work has started within IBM, to attempt to describe a number of industry standard data formats, including SWIFT and ISO 8583, using DFDL. Simon has a long standing work item to attempt to describe a number of custom application formats. These efforts should help validate the utility of DFDL and suggest specification revisions. Thanksgiving is next week - will this affect attendence of next weeks meeting? Meeting closed 17:35 GMT Actions Mike to comment on the omission of initial prefix Separators and final postfix Separators from the productions table, in the revised DFDL Data Grammar. Are these intended to be covered by other elements in the productions, or do they need to be added? References Object Constraint Language; specification at http://www.omg.org/technology/documents/formal/ocl.htm Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Call today - MikeB cannot attend
by Mike Beckerle 14 Nov '07

14 Nov '07

I have an unmovable conflict so I can't make the DFDL OGF call today. ...mikeb

1 0

Dfdl information set 7
by Simon Parker 12 Nov '07

12 Nov '07

Good morning. Version 7 of the proposal incorporates the outcome of last week's conference and subsequent email discussion. If I've overlooked or misunderstood any amendments, please let me know. Regards, Simon Simon Parker Software Consultant PolarLake 80 Harcourt Street Dublin 2, Ireland Phone +353 1 449-1075 Fax +353 1 449-1011 Web http://www.polarlake.com/ The information transmitted in this email is intended for the addressee only and may contain confidential and/or privileged material. Any review, retransmission, dissemination, reliance upon or other use of this information by persons or entities other than the addressee is prohibited. If you think, for any reason, that this message may have been addressed to you in error, we would ask you to notify the sender immediately by return email and delete the material. PolarLake Limited | Registered in Dublin, Ireland | Number 357324 | Registered office as above.

1 0

DFDL: *CORRECTED* Minutes from OGF WG call, 7 Nov 2007
by Ian W Parkinson 09 Nov '07

09 Nov '07

One correction in the revised minutes below: some duplicate text removed from para 2.1. With thanks to Simon for spotting the error. Ian Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 7 Nov 2007 Attendees Mike Beckerle (IBM) Geoff Judd (IBM) Steve Hanson (IBM) Suman Kalia (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) 1. Introduction The previous meeting left two areas for consideration: the Information Set (InfoSet) proposal, and the work regarding defaulting and nulls. This meeting would cover a review of a revised InfoSet proposal. The defaults/nulls issue has not been progressed. 2. Review of Revised Information Set Proposal Steve and Simon have collaborated on a revision, v6, of the InfoSet proposal. This has been circulated via the DFDL-WG mailing list. The meeting reviewed the v6 proposal, highlighting the following points: 1. There remain two types of information item: Element Information Items and Document Information Items. There are now two categories of Element Information Item: simple, and complex. 2. The description of Document Information Items incorrectly lists a "root" member. This should be replaced by a "children" member. Geoff pointed out that the associated UML diagram suggests that a Document Information Item may contain multiple children; this will be restricted in the text. 3. The "dfdl version" member is now used to describe the version of DFDL to which the InfoSet conforms; it does not describe the version of the parser generating the InfoSet. In particular, for an InfoSet to inlcude a dfdl version identifier, there is now no requirement for that InfoSet to have been generated by a DFDL parser. 4. Mike queried the meaning of "absolute Schema Component Descriptor"; it should identify a schema as a whole. This is a term defined by the SCD specification, and an appropriate citation should be added to the proposal. 5. The meeting discussed, at length, the "data value" member of Element Information Item, which the proposal lists as being of type String. Mike pointed out that this is insufficient to distinguish between some binary floating-point values, and suggested that it should be of a type defined by the "data type" member. Suman suggested this would also make for a more natural API. Simon remarked on the precedent set by XML Schema where all values are represented as strings. Mike closed the discussion advocating dropping the word "String" from the definition of the "data value" member. 6. Suman queried the purpose of the document information item. This is an in-memory artifact only, which acts as a container of the InfoSet's data, and has no representation in a serialization of the InfoSet. This should be clarified in the proposal. 7. The proposal now contains a UML diagram illustrating the object model. The relationship between Parent and Element requires some clarifying: it should be modelled as a composite containment relationship. 8. Simon highlighted that the "schema" member of Element Information Item may be either a relative or an absolute SCD; while the "schema" member of Document Information Item must be an absolute SCD. 9. The current version makes no mention of arrays, preferring to treat arrays as repeated children with the same name. The meeting consdered how to denote arrays in the InfoSet, but opted instead to add a note regarding arrays to the specification. Mike will circulate his notes regarding the agreed changes to the spec. Simon will circulate a new version, v7, incorporating further comments. 3. Other Business There was no other business. Meeting closed Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL: Minutes from OGF WG call, 7 Nov 2007
by Ian W Parkinson 08 Nov '07

08 Nov '07

Hi all, Steve and Mike have asked me to record minutes at the regular DFDL-WG conference calls. Here are the minutes from yesterday's meeting - I'd obviously welcome any suggestions you may have on the format or style of these minutes. Thanks, Ian Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 7 Nov 2007 Attendees Mike Beckerle (IBM) Geoff Judd (IBM) Steve Hanson (IBM) Suman Kalia (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) 1. Introduction The previous meeting left two areas for consideration: the Information Set (InfoSet) proposal, and the work regarding defaulting and nulls. This meeting would cover a review of a revised InfoSet proposal. The defaults/nulls issue has not been progressed. 2. Review of Revised Information Set Proposal Steve and Simon have collaborated on a revision, v6, of the InfoSet proposal. This has been circulated via the DFDL-WG mailing list. The meeting reviewed the v6 proposal, highlighting the following points: There remain two types of information item: Element Information Items and Document Information Items. Element Information Items. There are now two categories of Element Information Item: simple, and complex. The description of Document Information Items incorrectly lists a "root" member. This should be replaced by a "children" member. Geoff pointed out that the associated UML diagram suggests that a Document Information Item may contain multiple children; this will be restricted in the text. The "dfdl version" member is now used to describe the version of DFDL to which the InfoSet conforms; it does not describe the version of the parser generating the InfoSet. In particular, for an InfoSet to inlcude a dfdl version identifier, there is now no requirement for that InfoSet to have been generated by a DFDL parser. Mike queried the meaning of "absolute Schema Component Descriptor"; it should identify a schema as a whole. This is a term defined by the SCD specification, and an appropriate citation should be added to the proposal. The meeting discussed, at length, the "data value" member of Element Information Item, which the proposal lists as being of type String. Mike pointed out that this is insufficient to distinguish between some binary floating-point values, and suggested that it should be of a type defined by the "data type" member. Suman suggested this would also make for a more natural API. Simon remarked on the precedent set by XML Schema where all values are represented as strings. Mike closed the discussion advocating dropping the word "String" from the definition of the "data value" member. Suman queried the purpose of the document information item. This is an in-memory artifact only, which acts as a container of the InfoSet's data, and has no representation in a serialization of the InfoSet. This should be clarified in the proposal. The proposal now contains a UML diagram illustrating the object model. The relationship between Parent and Element requires some clarifying: it should be modelled as a composite containment relationship. Simon highlighted that the "schema" member of Element Information Item may be either a relative or an absolute SCD; while the "schema" member of Document Information Item must be an absolute SCD. The current version makes no mention of arrays, preferring to treat arrays as repeated children with the same name. The meeting consdered how to denote arrays in the InfoSet, but opted instead to add a note regarding arrays to the specification. Mike will circulate his notes regarding the agreed changes to the spec. Simon will circulate a new version, v7, incorporating further comments. 3. Other Business There was no other business. Meeting closed Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL Information Set proposal 6
by Simon Parker 07 Nov '07

07 Nov '07

Good afternoon. Steve and I have collaborated to compose this revised proposal (dfdl-infoset-proposal.6.doc) for the DFDL information set. I attached the diagram source representation (dfdl_infoset.3.uxf) so anyone can produce fresh versions. I used a simple, free Eclipse plugin called 'Umlet <http://www.umlet.com/> '. Comments and questions are welcome, as ever. Simon Simon Parker Software Consultant PolarLake 80 Harcourt Street Dublin 2, Ireland Phone +353 1 449-1075 Fax +353 1 449-1011 Web http://www.polarlake.com/ The information transmitted in this email is intended for the addressee only and may contain confidential and/or privileged material. Any review, retransmission, dissemination, reliance upon or other use of this information by persons or entities other than the addressee is prohibited. If you think, for any reason, that this message may have been addressed to you in error, we would ask you to notify the sender immediately by return email and delete the material. PolarLake Limited | Registered in Dublin, Ireland | Number 357324 | Registered office as above.

1 0

DFDL: Minutes from OGF WG call 2007-10-31
by Mike Beckerle 01 Nov '07

01 Nov '07

OGF DFDL WG Call 2007-10-31 Discussed: Separator semantics via rewriting - Alan Powell (prior to call) raised objections to MikeB's treatment of separator properties semantics in draft of nulls/optionals/defaults memo. Mike agrees. This stuff is a mess. Needs to be reworked. Action item: Mike to rework. Discussed: Boxed data, i.e., sequences with dfdl:length specifications as means of specifying box size surrounding dfdl-described content. Proposed to drop this from DFDL V1.0. Action: examine important formats to see if this feature is needed. We know there are formats that use it (IMS blocks), but must they be supported in single schemas or is a two-layer via two schemas approach acceptable? Resolved: not dropping this. New simpler examples are needed which make this clearer than the example in the email/memo by Mike. (Note: this is the whole section on valueCalc properties. This has already been identified as an area needing a write up.) Discussed: does infoset include things like the representation length of items? This information is needed to express output for formats where one element stores the rep-length of another. Suman pointed out that this information doesn't need to be available in the infoset. The relationships between elements of many kinds cannot be expressed directly in terms of the infoset, so why require this representation-level relationship to be so expressed? Resolved: infoset does not contain represnetation artifacts like representation length. Mike mentioned that it does contain number of occurrences for arrays. However, later discussion of infoset brings into doubt whether the infoset even knows about arrays. Discussed: infoset Metadata link? Suman has advocated for this. It's an optional thing. SCD paths are acceptable to most. Simon maintained that this shouldn't be part of the infoset but really is an implementation consideration. UML model: Discussed that it should match XML infoset terminology and also the grammar terminology used in the spec. (note: or the grammar terminology be made to conform to the infoset - actually this is more likely) Simon's model has arrays as distinct from sequence groups. We discussed eliminating arrays as a concept and going to a neutral term that captures both sequence groups and arrays (things with children). Action: Steve H. and Simon P. to work out remaining infoset issues. Provide UML diagram. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

1 0