June 2006 - dfdl-wg - lists.ogf.org

DFDL scoping version 018
by Mike Beckerle 07 Jul '06

07 Jul '06

I incorporated comments from our WG call and updated the scoping document: https://forge.gridforum.org/sf/docman/do/viewDocument/projects.dfdl-wg/docm… There is no substantive change here. I just put in the TBDs people asked for, fixed a few things where extensibility was being impacted, and fixed typos. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Integration Solutions Westborough, MA 01581 voice and FAX 508-599-7148 home/mobile office 508-915-4795

2 2

DFDL WG call 2006-06-28
by Mike Beckerle 28 Jun '06

28 Jun '06

Attendees: Geoff Judd, Bob McGrath, Mike Beckerle We discussed the "consolidated" document, w.r.t. folding in missing sections. We discussed that updating the sections on conversions, and extensions should wait for current work on conversions and groups to draw some conclusion. We discussed that the arrays material needs to be segmented to fit the outline of basic material vs. detailed material. Bob said he'll try to do that or at least make notes in it about areas where there is uncertainty about where a section should go. Geoff will check with Steve Hanson about folding in the properties document. The latest IBM-internal version of this document needs at least to be stripped of IBM-specific material before we publish it to the WG. The big properties list will again raise the issue of how to segment into core + extra packages, and what is core and what is extension. That's ok, we need to have that discussion. There are several sections such as the work Geoff lead on optionals/default/nullable, etc. and also on wildcard behavior that need to get folded in, but without the properties to refer to it's very unclear how to put these sections in. They are in some sense about the meanings of specific groups of properties and their interactions. Question for the group: does anyone have anything written about layering and hidden layers that is newer than last september's "draft" document. There was material there about layering that I did not bring forward into the "consolidated" document yet because I wasn't sure if it was still current or not.. Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Integration Solutions Westborough, MA 01581 voice and FAX 508-599-7148

1 0

DFDL Telcon tomorrow
by Westhead, Martin (Martin) 28 Jun '06

28 Jun '06

Hi folks, a reminder for tomorrow - I will not be there (I am on a training course). Martin Time: 9am PDT Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call.

1 0

Telcon notes 2006-06-21
by Westhead, Martin (Martin) 21 Jun '06

21 Jun '06

Present Martin Westhead Bob McGrath Mike Beckerle Geoff Judd Steve Hanson Discussion We reviewed status: * Geoff and Martin have been discussing the issues of separators at different levels * Mike has been working on bringing the concepts of properties and variables together * We still have an unassigned action to assemble a draft document from the current pieces - Mike is going to make a start on this. * There are outstanding edits to the arrays document that Bob is ready to make he is waiting for the outline document. Note: Martin will not be on the next 2 calls due to training and vacation.

1 0

Telcon now
by Westhead, Martin (Martin) 21 Jun '06

21 Jun '06

Conference call instructions: Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call. Web Conference: https://webconferencing.avaya.com <https://webconferencing.avaya.com/> Web Conference: Part. Code 785343; WebConf Reference 158672 Spec document outline: Introduction section * glossary * What is DFLD? * Model Language basics * syntax * scoping and context * guards and selectors on annotations * basic arrays * variables * layering (hidden layers and value calc/referencing) Detailed semantics * properties and conversions Arrays Built-in specifications Extensions

1 0

DFDL Telcon notes
by Westhead, Martin (Martin) 14 Jun '06

14 Jun '06

Present Mike Beckerle Martin Westhead Geoff Judd Bob McGrath Discussion This was a very useful discussion. We started by formulating the outline of the spec document and then reviewed some specific aspects of the new format document. Spec document outline: Introduction section * glossary * What is DFLD? * Model Language basics * syntax * scoping and context * guards and selectors on annotations * basic arrays * variables * layering (hidden layers and value calc/referencing) Detailed semantics * properties and conversions Arrays Built-in specifications Extensions Extensibility - constructs like <dfdl:sequence> need to have their list of allowed properties extensible Format/defineFormat - is the dfdl:format tag correctly named. Martin assumed a catch-all dfdl annotation tag. Mike assumed that dfdl:format was *just* format and that other things (variable assignments, conversion definitions etc.) would go in different tags. More examples and investigation are required to see how this works in practice. defineFormat is probably too specific right now, this is an important reuse mechanism and it should be available to apply to all top level dfdl tags. One solution would be to have a more generic "define" tag that can contain "dfdl:format" tags along with others. Properties and variables - we expect that properties and variables have the same scoping, assignment and reference semantics. Properties are only different in the sense that they represent parameters to conversions. Linked properties - we discussed the need for linked properties, it seems like extra stuff that might not be required and it was useful to see if there were different ways to achieve the same goal. From the discussion it did not seem an unreasonable thing to include. Applies default - we reviewed the issues around the "applies" default (toScope or hereOnly) the current default is "hereOnly" different examples suggest different defaults are more appropriate. We decided to leave the decision where it was but a bigger selection of examples will probably help inform this decision. Actions * Assemble draft of spec document from the pieces that we have. (Currently unassigned - volunteers?)

1 0

Telcon Now
by Westhead, Martin (Martin) 14 Jun '06

14 Jun '06

(Sorry for the late notice) Main item for the agenda is again Mike's scoping document. Conference call instructions: Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call. https://webconferencing.avaya.com/default.htm?ConfRef=158671&Pin=139469

1 0

Telcon notes 2006-06-07
by Westhead, Martin (Martin) 07 Jun '06

07 Jun '06

Present Martin Westhead Mike Beckerle Steve Hanson Suman Kali Bob McGrath Discussion We had a short discussion where Mike gave a brief overview of some of the features of the new scoping document. We agreed on two agenda items for next week: * Detailed discussion of scoping * Assembling a complete spec document: what pieces are required? What work needs to be done? Deadline to present at GGF in DC in October.

1 0

Telcon tomorrow
by Westhead, Martin (Martin) 07 Jun '06

07 Jun '06

Main item for the agenda is Mike's scoping document. Conference call instructions: Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call. Web Conference: https://webconferencing.avaya.com <https://webconferencing.avaya.com/> Web Conference: Part. Code 785343; WebConf Reference 158672

1 0

Defuddle Questions and Pull-Parsing Thoughts
by Tom Sugden 06 Jun '06

06 Jun '06

Hi all, Apologies for missing the telcon this week due to other work pressures. I haven't made much progress with any implementation, but have been taking a look at the Defuddle code. I have some questions that someone may be able to answer, and a few thoughts for discussion. The current Defuddle implementation is based upon JAXME, the Java/XML binding implementation. Presumably JAXME is used to generate an object model representation of the data format described by the DFDL schema. And then, I think the underlying data stream would be marshaled into an instance of that object model. Is this understanding correct? If my understanding is correct, I'm concerned that this approach may not be suitable for large data streams, since the entire object model instance would probably have to be assembled and stored in memory, like a DOM tree. Has anybody considered using a streamed pull-parsing approach instead, based upon or similar to StAX (Streaming API for XML)? I was thinking along the lines of parsing the DFDL schema into DOM or some other internal representation. Then pull-parsing the data stream, producing a sequence of StAX-like events corresponding to the data in the stream and its structure. During the pull-parsing, the context would need to be maintained and the conversion algorithm used for transforming parts of the data stream into values of the correct type. These values would then be wrapped in corresponding event objects. If this approach was viable, then these StAX-like APIs could be used to implement higher-level applications or APIs. For instance, it would be straight-forward to produce an XML serialization of any data described by a DFDL schema. One could also imagine binding any data described by a DFDL schema to auto-generated Java beans, or to a DOM object, when desirable. The process may even be reversible, so that data could be written back to a data stream as well as being read from one. I haven't thought this through very deeply yet and my understandings of the issues are still quite naive, so I will be very interested to hear any comments. Sorry if this avenue has already been explored, or I've misunderstood the mechanics of Defuddle or JAXME. Cheers, Tom

2 1