- dfdl-wg - lists.ogf.org

Telcon Notes 2006-3-22
by Westhead, Martin (Martin) 22 Mar '06

22 Mar '06

Present: Martin Westhead Bob McGrath Steve Hanson Geoff Judd Reviewed Geoff's document on "Support of Uncertainty". This is looking in good shape. Considered Steve's email on escape characters. It was observed that we need to have a way to handle XML escaping although this is a sufficiently important special case that perhaps it can be handled just as a stand alone special case. Jim raised the issue of DFDL implementations and suggested that moving forward he was going to try to focus his DFDL time on driving implementations rather than on all the language details. It was generally agreed that bringing Defudle up to date would be a big contribution to the group. Coming soon: Nulls - Geoff Properties doc - Steve

1 0

Telcon Tomorrow
by Westhead, Martin (Martin) 22 Mar '06

22 Mar '06

Sorry I missed last week and so I'm a little out of the loop I have no agenda items at this point. Contributions welcomed? Conference call instructions: Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call. Web Conference: https://webconferencing.avaya.com <https://webconferencing.avaya.com/> Web Conference: Part. Code 785343; WebConf Reference 158672

3 2

Transformation between Binary data and XML
by Mike Beckerle 20 Mar '06

20 Mar '06

FYI: "Edmund Lai" <edmund.lai(a)gmail.com> 03/20/2006 02:37 AM To Mike Beckerle/Worcester/IBM@IBMUS cc Subject Transformation between Binary data and XML I have done some experiment on transformation between binary data and XML that maybe of interest to DFDL-WG. I am not following any existing spec, but rather I am trying to determine a set of commands that I feel is adequate and easy to learn. Only after I finish would I try to see it I can modify it to fit into any standard. The converter is web based and is in http://www.datamech.com/XMLForm/XMLandBinary.html The description is in http://www.datamech.com/XMLForm/aboutXmlAndBinary.html

1 0

How to deal with variable length elements?
by Robert E. McGrath 17 Mar '06

17 Mar '06

Folks, I'm trying to build up a story about how to handle arrays using DFDL. But first, I need to check if I understand the basics. Here is an example of a 1D array in XML, modeled as two elements, an integer indicating how many elements, followed by an array of zero or more floats. Looking at my XML textbooks, the following seems like the correct XML schema for this notion. +++  <xs: element type="float" name="floatType" \>  <xs:complexType name="floatArray"> <xs:sequence> <xs:element name="nelems type="int" /> <xs:element name="x" ref="floatType" minOccurs="0" maxOccurs="./nelems" /> </xs:sequence> </xs:complexType> +++ Do I have this correct? (I'm pretty sure the 'maxOccurs' isn't correct, so I hope someone will tell me the right way to do this.) If so, the next question will be "how do I annotate this with DFDL?", e.g., when the data is precisely one binary (or text encoded) int, followed by some binary (or text encoeded) floats.

3 5

RE: [dfdl-wg] Telcon tomorrow
by Steve Hanson 15 Mar '06

15 Mar '06

As discussed on the call, I'll issue the first draft of the properties without creating separator/terminator/initiator schemes, and we'll decide when we have evaluated the full property set. Regards, Steve Steve Hanson WebSphere Message Brokers, IBM United Kingdom Ltd, Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Steve Hanson/UK/IBM To 09/03/2006 10:16 "Westhead, Martin \(Martin\)" <westhead(a)avaya.com> cc dfdl-wg(a)ggf.org, owner-dfdl-wg(a)ggf.org Subject RE: [dfdl-wg] Telcon tomorrow (Document link: Steve Hanson) As it currently stands, the following properties apply to separators and there may well be more (given there is Encoding there should probably be byteOrder & BOM properties for Unicode). separator separatorEncoding finalSeparatorCanBeMissing separatorPosition separatorPolicyForAbsentElements Maybe it makes more sense to treat separators like quotes and have separator schemes that the user can define and re-use? Regards, Steve Steve Hanson WebSphere Message Brokers, IBM United Kingdom Ltd, Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 "Westhead, Martin \(Martin\)" <westhead(a)avaya.c To om> Steve Hanson/UK/IBM@IBMGB cc 08/03/2006 16:38 <dfdl-wg(a)ggf.org>, <owner-dfdl-wg(a)ggf.org> Subject RE: [dfdl-wg] Telcon tomorrow That seems sensible. Question: Can you/should you be able to extend sub properties. For example I introduce a new conversion based on the existing separator function that adds a tag prefix to each field...I would like to be able to add (when I introduce this conversion) a new sub-property to separator called "tagPrefix". Martin >-----Original Message----- >From: Steve Hanson [mailto:smh@uk.ibm.com] >Sent: Wednesday, March 08, 2006 2:07 AM >To: Westhead, Martin (Martin) >Cc: dfdl-wg(a)ggf.org; owner-dfdl-wg(a)ggf.org >Subject: Re: [dfdl-wg] Telcon tomorrow > >Hi Martin > >Additional agenda item. This is something us IBMers discussed prior to the >F2F but I omitted to include it in my foils about properties. > >Several properties have a bunch of other sub-properties associated with >them. For example, separator has separatorEncoding and separatorIgnoreCase; >initiator and terminator likewise have equivalent sub-properties. It was >suggested that if sub-properties needed to be specified they should only be >specifiable on a sub-element, as below. This was to reduce the number of >properties on DFDL elements. > ><dfdl:sequence separator="@" /> > > or > ><dfdl:sequence> > <dfdl:separator encoding="850" ignoreCase="true">@</dfdl:separator> ></dfdl:sequence> > >Regards, Steve > >Steve Hanson >WebSphere Message Brokers, >IBM United Kingdom Ltd, Hursley, UK >Internet: smh(a)uk.ibm.com >Phone (+44)/(0) 1962-815848 > > > > "Westhead, Martin > (Martin)" > <westhead(a)avaya.c To > om> <dfdl-wg(a)ggf.org> > Sent by: cc > owner-dfdl-wg@ggf > .org Subject > [dfdl-wg] Telcon tomorrow > > 07/03/2006 22:03 > > > > > > > > >Preliminary Agenda >------------------ > > 1. Review outcome of F2F meeting > 2. Planning for next GGF > > >Conference call instructions: >Number: 1-800-664-0771 >At the automated answering system, press 0 for the operator. >Ask for the "Daffodil working group" call. > >Web Conference: https://webconferencing.avaya.com >Web Conference: Part. Code 785343; WebConf Reference 158672 >

1 0

Document on modelling uncertainty
by Geoff Judd 15 Mar '06

15 Mar '06

Hi, I have attached a document that discusses the support of uncertainty within DFDL. It is an extension of the choices document that was discussed at the last F2F but now includes choices, element wildcards, optionality, substitution groups and late binding of DFDL Schemas. There is an Alternatives Considered section at the end which contains ideas that were considered but rejected. Regards, Geoff Judd Websphere MQ Integrator Development IBM United Kingdom Ltd. Hursley Telephone: +44-1962-818461 E-mail: JUDDG(a)uk.ibm.com

1 0

RE: [dfdl-wg] Telcon tomorrow
by Steve Hanson 09 Mar '06

09 Mar '06

As it currently stands, the following properties apply to separators and there may well be more (given there is Encoding there should probably be byteOrder & BOM properties for Unicode). separator separatorEncoding finalSeparatorCanBeMissing separatorPosition separatorPolicyForAbsentElements Maybe it makes more sense to treat separators like quotes and have separator schemes that the user can define and re-use? Regards, Steve Steve Hanson WebSphere Message Brokers, IBM United Kingdom Ltd, Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 "Westhead, Martin \(Martin\)" <westhead(a)avaya.c To om> Steve Hanson/UK/IBM@IBMGB cc 08/03/2006 16:38 <dfdl-wg(a)ggf.org>, <owner-dfdl-wg(a)ggf.org> Subject RE: [dfdl-wg] Telcon tomorrow That seems sensible. Question: Can you/should you be able to extend sub properties. For example I introduce a new conversion based on the existing separator function that adds a tag prefix to each field...I would like to be able to add (when I introduce this conversion) a new sub-property to separator called "tagPrefix". Martin >-----Original Message----- >From: Steve Hanson [mailto:smh@uk.ibm.com] >Sent: Wednesday, March 08, 2006 2:07 AM >To: Westhead, Martin (Martin) >Cc: dfdl-wg(a)ggf.org; owner-dfdl-wg(a)ggf.org >Subject: Re: [dfdl-wg] Telcon tomorrow > >Hi Martin > >Additional agenda item. This is something us IBMers discussed prior to the >F2F but I omitted to include it in my foils about properties. > >Several properties have a bunch of other sub-properties associated with >them. For example, separator has separatorEncoding and separatorIgnoreCase; >initiator and terminator likewise have equivalent sub-properties. It was >suggested that if sub-properties needed to be specified they should only be >specifiable on a sub-element, as below. This was to reduce the number of >properties on DFDL elements. > ><dfdl:sequence separator="@" /> > > or > ><dfdl:sequence> > <dfdl:separator encoding="850" ignoreCase="true">@</dfdl:separator> ></dfdl:sequence> > >Regards, Steve > >Steve Hanson >WebSphere Message Brokers, >IBM United Kingdom Ltd, Hursley, UK >Internet: smh(a)uk.ibm.com >Phone (+44)/(0) 1962-815848 > > > > "Westhead, Martin > (Martin)" > <westhead(a)avaya.c To > om> <dfdl-wg(a)ggf.org> > Sent by: cc > owner-dfdl-wg@ggf > .org Subject > [dfdl-wg] Telcon tomorrow > > 07/03/2006 22:03 > > > > > > > > >Preliminary Agenda >------------------ > > 1. Review outcome of F2F meeting > 2. Planning for next GGF > > >Conference call instructions: >Number: 1-800-664-0771 >At the automated answering system, press 0 for the operator. >Ask for the "Daffodil working group" call. > >Web Conference: https://webconferencing.avaya.com >Web Conference: Part. Code 785343; WebConf Reference 158672 >

1 0

Notes from Telcon 2006-3-8
by Westhead, Martin (Martin) 08 Mar '06

08 Mar '06

Present Martin Westhead Robert McGrath Steve Hanson Geoff Judd Tara Talbott Choices Geoff is working on the choices document. He has suggestions about how Schematron might be incorporated. We may see something before next week's session Separator We discussed some of the issues with the separator logic. Fundamentally we have formats that cannot be generally handled using the simplistic pipeline model that we have. In order to maintain modularity we are considering if there is a way to specify the separation semantics in a way orthogonal to the pipeline (i.e. bolted on the side). GGF It appears that no one from the main group of participants will be attending the next GGF in Tokyo. Since we were not present at the last GGF we may need to do something. A possibility would be to have someone we know (e.g. OGSA-DAI team) host the meeting and call in to give a remote presentation. (BTW - I may not be able to make next weeks call)

1 0

Telcon tomorrow
by Westhead, Martin (Martin) 08 Mar '06

08 Mar '06

Preliminary Agenda ------------------ 1. Review outcome of F2F meeting 2. Planning for next GGF Conference call instructions: Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call. Web Conference: https://webconferencing.avaya.com <https://webconferencing.avaya.com/> Web Conference: Part. Code 785343; WebConf Reference 158672

2 1

Notes from Face to Face day 2 and 3
by Westhead, Martin (Martin) 03 Mar '06

03 Mar '06

Present Martin Westhead Steve Hanson Suman Kalia Geoff Judd Actions The following actions were assigned: * Scoping and context [guards and tests] (Mike) * Properties and conversions (Steve) * Revisit conversions [Steve's questions, type matching, position, speculative parsing, where are we?](Martin) * Uncertainty [wildcards, substitution groups, choices, optionality] (Geoff) * Defaults and nulls (Geoff) * DFDL Schema (Suman) Open issues: * syntax (sequences, markup) * identifying core properties and conversions (if any) * round tripping * embedded XML Day 2 Notes Properties/conversions/grouping - Steve/Suman (Suman took notes on this topic) Steve will be re-writing the current document to take on board proposals from this meeting and to start the process (with Martin) of trying to converge the properties with the conversions material. Choices and wildcards - Geoff We explored several ideas around the framework that Geoff has. Geoff will be writing these up. Comments included: * Possible use of binhex to represent "deferred" choices * Modeled uncertainty with "any" should be treated as a choice * Substitution groups should be treated as a choice * Wildcards will be handled here and "open" wildcards (unmodeled uncertainty) could be handled by dropping in a string/binhex * Schematron-like assertions could be used as choice guards such assertions could also make general statements about data validity * Use of XML Schema validation to resolve speculative parsing raises issues. (Either all DFDL parsers must validate or would get different results) Which XML Schema constructs are supported - Steve We reviewed Steves slides made various modifications key decisions: * Attributes - wait and see Final documents We had an inconclusive discussion on the structure of the final documents. Day 3 Notes Instance Validation - Efficiency of XML Schema validation - Implementation difficult of XML Schema validation - Therefore cannot force the use of full DFDL parsing - so we cannot use XML Schema validation to resolve speculative parsing - property to state validation to be used could allow parser to produce a warning error if not available - possibility: could use require just facet/value checks (not structure) - XPath evaluation required - Therefore XPath assertions for choice discrimination possible - And XPath assertions for describing valid data possible - EDI community has strong requirement for such assertions to be available (such as coexistance rules) - Is there a requirement for content validation (since construction is driven by model) - Might be useful to have standard names for classes of error DFDL Schema Validation - XML Schema valid - Syntax and semantics of annotations (annotations must be consistent with schema) - XPath references are valid (standard should describe ways validation can be done and features that can be validated, validation should not be mandated).

1 0