- dfdl-wg - lists.ogf.org

Wednesday's telcon
by Martin Westhead 29 Sep '05

29 Sep '05

9am PDT Wednesday 27th September 2005 Preliminary Agenda ------------------ - F2F at GGF (any outstanding issues) - Primer use cases (?) - Issues list (starting from item 28) Conference call instructions: Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call.

2 1

Telcon notes 2005-09-28
by Martin Westhead 28 Sep '05

28 Sep '05

Present ======= Martin Westhead Tara Talbot Robert McGrath Geoff Judd Steve Hanson Jim Meyers Agenda ====== F2F at GGF (any outstanding issues) ----------------------------------- No issues raised at this time. Primer use cases (?) -------------------- Short discussion of use-cases. Existing examples seem complicated, Rob explained that this was just things he knew about that he'd like DFDL to do. Action: Martin and Steve agreed to produce some simple examples. Extensibility discussion ------------------------ Jim raised the topic of extensibility. He is concerned that aspects of the current proposal will not allow the sort of extensibility that he would like to see. Jim is suggesting that perhaps a separate group (possibly overlapping) should be exploring this issue. He would like this group to consider approaches that in the short term may require changes to what is currently on the table. Action [Martin]: Extensibility tracker add Steve and Suman Action [Mike]: Add Extensibility to F2F Agenda Choices and wildcards (Geoff) ----------------------------- This was not discussed due to lack of time and the fact that Mike was not available. Action [Martin]: On the agenda for next week. Issues list (starting from item 28) ----------------------------------- No progress - start from 28 next week.

1 0

New Primer: I need some help
by Robert E. McGrath 26 Sep '05

26 Sep '05

Greetings, As many of you know, I have been tasked to compile a revised "primer" for DFDL. Based on the draft spec. and other information, I have made some good progress, but there are a number of topics that I don't have enough information on. This note summarizes areas where I need substantial contributions from you all. This may not be things to tackle via the mail list, I don't know. Perhaps we can discuss these issues at the F2F. 1. Use cases Actually, there should be a "Use Cases" tracker and a separate UC document, IMO. By "Use Case" I mean statements of how DFDL is intended to be used. This is different from the current example suites which illustrate the language but do not discuss why you need DFDL or what it helps you do. More on this one later. I'm developing a note with some interesting use cases I'd like to discuss. 2. Overview of the language I would like the primer to say "the DFDL is the following XSD elements, with the following restrictions. DFDL defines the following tags and attributes". Just a summary, circa 2 pages. I'm sure this informtion is implicit in the current spec and demo software, but I need help to write a clear summary. 3. Processing model I would like the primer to explain how DFDL is used, i.e., Parse DFDL, which tells how to parse data, ... Again, this is probably implicit in current work, but I need a short and clear summary. 4. Explanation of configurations, layers etc. Related to above. Explain the non-trivial processing of DFDL. Some of this awaits the outcome of trackers.

2 1

Discussion document: some non-trivial uses of DFDL
by Robert E. McGrath 23 Sep '05

23 Sep '05

As I have been studying the DFDL and working on the primer, I have been thinking about how it might be used in Grid applications. A virtual walk around NCSA has produced quite a few non-trivial use cases. What I would like to do is put some of these out to the group, for discussion. Since I am relatively new to the DFDL game, I expect that some of my ideas are naive or flat wrong. I have typed up detailed notes on 4 cases. I have a couple of other cases I'm trying to document. I attach these notes here. Sorry, it's a bit long! -- --- Robert E. McGrath, Ph.D. National Center for Supercomputing Applications University of Illinois, Urbana-Champaign Urban, Illinois 61820 (217)-333-6549 mcgrath(a)ncsa.uiuc.edu

1 0

[Fwd: BOUNCE dfdl-wg@ggf.org: Non-member submission from ["Dave Berry" <daveb@nesc.ac.uk>]]
by Martin Westhead 23 Sep '05

23 Sep '05

Forwarded from Dave Berry, advertising a BOF on Data streams at GGF.

1 0

F2F logistics - IBM Copley Place offices
by Mike Beckerle 23 Sep '05

23 Sep '05

During the upcoming GGF15 we'll be having a concurrently scheduled F2F meeting. This meeting will be hosted by IBM at its Boston facility. I have booked Conference Room 101 for Monday, Tuesday, and Wednesday all day. This seats 10 and has a speakerphone, network connection, whiteboard. I have booked Conference Room 102 for Tuesday and Wednesday all day. This seats 4 and is for smaller side discussions breakouts. It also has a speakerphone, network connection, can't recall about whiteboard. I think so. I will bring a LCD projector for slide showing. Here's information on the physical location: This map shows both Copley place where IBM's office is, and the Park Plaza Hotel (where GGF15 is being held). http://maps.google.com/maps?q=Copley+Place.+Boston+MA&ll=42.348855,-71.0739… Here's a google earth image where I've put graffiti marking the Park Plaza Hotel where GGF15 is being held, and the Copley Place complex where IBM's offices are. It's 3.5 blocks walking. It is barely worth a cab ride unless it's raining. Don't bother with the subway either. The walks to/from the subway are the same distance as walking directly. Copley place has two buildings. We're in "2" Copley place, though I don't think your average person on the street knows which building is 1 and which is 2. Number one has a Westin hotel and some shopping. If you enter there you'll need to take the walkway across to the other building which has the larger bulk of the shopping, Neiman Marcus dept. store, and the Marriott hotel, oh...and the IBM offices. Here's another Google Earth picture showing the Copley Place Buildings, which the hotels are, and roughly where in the complex you'll find IBM's offices. Once you are in the correct building, go to the very center of Copley Place Mall and look for a 3-story Waterfall. Behind it are elevators. Take these to the 'Sky lobby' (which is up 2 levels) and at the reception desk ask for IBM. You may need a security pass from the building security which will take a few minutes as they scan your picture ID and make a picture temporary badge from it for you. Everyone - BE SURE TO BRING A PICTURE ID IBMers - BE SURE TO BRING YOUR BADGE AND A PICTURE ID Then go right and the IBM office is on the left all on the same level. Here's a very low quality floorplan of the Copley Place Mall. Not sure it's helpful, but it's all I could find online. http://www.simon.com/mall/images/floorplans/784Copley_PlaceMall.gif

1 0

Telcon notes 2005-9-21
by Martin Westhead 21 Sep '05

21 Sep '05

Present ======= Martin Westhead Tara Talbot Bob McGrath Geoff Judd Suman Kalia Items ===== Agenda for sessions/F2F at GGF ------------------------------ Is registration to GGF necessary? Probably not if you are only interested in DFDL then group members can probably skip the GGF altogether. 3 day meeting Monday, Tuesday, Wednesday: GGF meetings o Monday 1100 DFDL intro session (Park Plaza) o Monday 1400 DFDL discussion session (Park Plaza) Topics (Mike will arrange them) o Summary of approach (Mike) o Summary of issues on major topics (Topic holders) o Arrays (Bob) o Scoping (Mike) o Layers (Tara) o Null and Default (Geoff) o Properties discussion (Suman) o Process (documents and timescales) (Martin) o Conclusions and priorities o Issues List (remaining time) Note: This is a reasonable list of topics based primarily on the list of trackers at the start of the meeting (of which there were 4). During the meeting we added 6 new trackers. In light of this we may want to revisit this list of topics next week and consider the priority of the new items against the old. - Martin Tracking document modifications ------------------------------- "Edit required" status added to issues list to denote items that have been discussed but for which document edits are still outstanding. Issues list (starting from item 16) ----------------------------------- Resolved 16-27 (list has been updated). Start next week on item 28. New trackers: - Properties discussion (Mike, Suman) - Open content and wild cards (Geoff, Steve) - Choice and guards (Geoff) - Use statements (Mike) - Extensibility (Martin, Tara, Jim, Mike, Bob) - Expression language (Mike, Suman, Martin) N.B. we did not discuss priorities on the call. All these tracker items have been set to medium.

1 0

Wednesday's telcon
by Martin Westhead 21 Sep '05

21 Sep '05

9am PDT Wednesday 21st September 2005 Preliminary Agenda ------------------ - Agenda for sessions/F2F at GGF - Tracking document modifications - Issues list (starting from item 16) Conference call instructions: Number: 1-800-664-0771 At the automated answering system, press 0 for the operator. Ask for the "Daffodil working group" call.

1 0

minutes of call 2005-09-14
by Mike Beckerle 20 Sep '05

20 Sep '05

Who: Jim Myers, Mike Beckerle, Tara Talbott, Steve Hanson, Geoff Judd, Bob McGrath Agenda: GGF plans - we will have F2F at GGF15. The schedule of our WG sessions at the GGF15 are not yet set, but we'll have F2F meetings during the non-scheduled times. We're counting on the fact that most of our DFDL WG members are focused on DFDL and won't mind the conflict with the rest of GGF15 much. People should make travel plans for all 3 days M, T, W. GridForge Forums - these seem to be working now. Will try them when one of our subcommittees reports back to the broader WG. E.g., scoping or arrays. Issues list - we made it through items 5 to 15 Issue 5 - resolution. No built-in set of defaults. If you need a property and your configuration doesn't have one specified then it is an error. There is a small set of named configurations provided with DFDL each of which is a self-consistent set of properties. This is probably the set that we find useful in the primer and other WG docs. Issue 6 - resolution. change attribute name from 'base' to 'extends' Issue 7 - belongs on same tracker as 8, 9, 10 Issue 11 - resolution: add an attribute 'byteOrderMarkPolicy' values are: required, notAllowed, optionalButGenerateOnOutput, optionalButOmitOnOutput (the generate on output control aspect was not discussed on the call. I thought of that just now while typing.) Issue 12 - fix diagram - omission of these types was unintentional. Better diagram next draft Issue 13 and 14 - new tracker item - Steve Hanson, Geoff Judd and someone from MikeB's team at IBM to address Issue 15 - resolution: each delimiter will grow an extra rep-property for its regexp variant. E.g., separator will have corresponding separatorRegexp property. Only one of the two may be specified. We discussed that separator (or any delimiter) can be a text string, and we had previously decided that the separatorEncoding attribute would go away to be replaced by a syntax for expressing hex bytes (not hex character codes, real non-charset transformed hex bytes) as part of delimiter strng literals. This same way of putting hex bytes would also be supported as part of the regexp language so that the regexp language remains able to include any non-regular expression for a delimiter into a regular expression. (Editor's Note: this is a bit tricky. We now have in string literals 3 different escape mechanisms that are different. 1) XML character code specifiers e.g., 'abc,def' which is the normal XML way of specifying the unicode #x2C character code. (This is an XML standard escape convention.) Used inside a delimiter string this means take the unicode #x2c code point, figure out what character it corresponds to in the charset of the data element and use that character. Note that this is the same thing that 'a' means. Take the 'a' unicode code point, figure out what character corresponds to 'a' in the charset of the data element, and look for that. 2) nonXML character code specifiers e.g., 'abc\#x00;def' which is the way we allow you to put XML-disallowed unicode character codes like #x00 into a string literal. (This is a DFDL convention). The interpretation here is exactly like the above. I.e., '\#x2c;' is exactly equivalent to ',' and the charset mapping applies as above. This rule is only needed because of the XML restriction disallowing certain unicode character codes from the XML infoset. 3) non-character byte specifiers. E.g., 'abc\%01;\%02;def' which is a proposed escape syntax that means put the bytes #x01 and #x02 into the string literal bypassing any considerations of charset, i.e., without considering them to be character codes in any character set. That is, these byte values have nothing to do with unicode codepoint values. (This is a DFDL convention.) The other characters of the string would be treated as per (1) and/or (2) above. )

2 2

Fw: [dfdl-wg] minutes of call 2005-09-14
by Steve Hanson 15 Sep '05

15 Sep '05

Hi Mike Re the three escape mechanisms. It is unfortunate that we need a separate mechanism for the second case (as discussed in the draft spec section 26). Re the third mechanism for escaping hex literals. An alternative syntax could be \#0x2C; using 0x to indicate hex. &#0x2C; would be even better - but does it cause XML problems? Regards, Steve ----- Forwarded by Steve Hanson/UK/IBM on 15/09/2005 09:33 ----- Mike Beckerle <beckerle(a)us.ibm. com> To Sent by: dfdl-wg(a)gridforum.org owner-dfdl-wg@ggf cc .org Subject [dfdl-wg] minutes of call 14/09/2005 22:18 2005-09-14 Who: Jim Myers, Mike Beckerle, Tara Talbott, Steve Hanson, Geoff Judd, Bob McGrath Agenda: GGF plans - we will have F2F at GGF15. The schedule of our WG sessions at the GGF15 are not yet set, but we'll have F2F meetings during the non-scheduled times. We're counting on the fact that most of our DFDL WG members are focused on DFDL and won't mind the conflict with the rest of GGF15 much. People should make travel plans for all 3 days M, T, W. GridForge Forums - these seem to be working now. Will try them when one of our subcommittees reports back to the broader WG. E.g., scoping or arrays. Issues list - we made it through items 5 to 15 Issue 5 - resolution. No built-in set of defaults. If you need a property and your configuration doesn't have one specified then it is an error. There is a small set of named configurations provided with DFDL each of which is a self-consistent set of properties. This is probably the set that we find useful in the primer and other WG docs. Issue 6 - resolution. change attribute name from 'base' to 'extends' Issue 7 - belongs on same tracker as 8, 9, 10 Issue 11 - resolution: add an attribute 'byteOrderMarkPolicy' values are: required, notAllowed, optionalButGenerateOnOutput, optionalButOmitOnOutput (the generate on output control aspect was not discussed on the call. I thought of that just now while typing.) Issue 12 - fix diagram - omission of these types was unintentional. Better diagram next draft Issue 13 and 14 - new tracker item - Steve Hanson, Geoff Judd and someone from MikeB's team at IBM to address Issue 15 - resolution: each delimiter will grow an extra rep-property for its regexp variant. E.g., separator will have corresponding separatorRegexp property. Only one of the two may be specified. We discussed that separator (or any delimiter) can be a text string, and we had previously decided that the separatorEncoding attribute would go away to be replaced by a syntax for expressing hex bytes (not hex character codes, real non-charset transformed hex bytes) as part of delimiter strng literals. This same way of putting hex bytes would also be supported as part of the regexp language so that the regexp language remains able to include any non-regular expression for a delimiter into a regular expression. (Editor's Note: this is a bit tricky. We now have in string literals 3 different escape mechanisms that are different. 1) XML character code specifiers e.g., 'abc,def' which is the normal XML way of specifying the unicode #x2C character code. (This is an XML standard escape convention.) Used inside a delimiter string this means take the unicode #x2c code point, figure out what character it corresponds to in the charset of the data element and use that character. Note that this is the same thing that 'a' means. Take the 'a' unicode code point, figure out what character corresponds to 'a' in the charset of the data element, and look for that. 2) nonXML character code specifiers e.g., 'abc\#x00;def' which is the way we allow you to put XML-disallowed unicode character codes like #x00 into a string literal. (This is a DFDL convention). The interpretation here is exactly like the above. I.e., '\#x2c;' is exactly equivalent to ',' and the charset mapping applies as above. This rule is only needed because of the XML restriction disallowing certain unicode character codes from the XML infoset. 3) non-character byte specifiers. E.g., 'abc\%01;\%02;def' which is a proposed escape syntax that means put the bytes #x01 and #x02 into the string literal bypassing any considerations of charset, i.e., without considering them to be character codes in any character set. That is, these byte values have nothing to do with unicode codepoint values. (This is a DFDL convention.) The other characters of the string would be treated as per (1) and/or (2) above. )

2 1