- dfdl-wg - lists.ogf.org

DFDL: Minutes from OGF WG call, 09 Jan 2007
by Ian W Parkinson 10 Jan '08

10 Jan '08

Open Grid Forum: Data Format Description Language Working Group Weekly Working Group Conference Call 17:00 GMT, 09 Jan 2008 Attendees Mike Beckerle (Oco) Geoff Judd (IBM) Simon Parker (PolarLake) Ian Parkinson (IBM) Alan Powell (IBM) Steve Hanson (IBM) Suman Kalia (IBM) Agenda 1. OGF 22 in Cambridge, MA 2. Level set on specification drafts 3. Expression Language 4. Nulls and defaults - can we drop useNullForDefault? 5. Other business 1. OGF22 The next OGF conference will be held February 25-29 in Cambridge, MA. As he is local, Mike is planning to attend to represent DFDL. The working group should decide what we would like to present at the conference, if anything, and Mike will enquire upon the closing date for submissions. Could be Jan 11th? 2. Specification Drafts Mike circulated draft 30 of the DFDL specification before Christmas, and had prepared a plan covering the contents of the next three drafts. The objective of the plan was to guide the group to the stage where the specification was not a limiting factor to progress and that implementations could proceed with a reasonable expectation that the specification would not change significantly. Steve mentioned that IBM are attempting to assign remaining workitems internally, and wanted to coordinate this with the other working group members to avoid duplication of effort. Due to the demands of his new role, Mike will need to pass some items that he had been hoping to tackle on to other people. He suggested that editorship of the specification should pass around the group with each draft, ideally to whoever would be making the most significant changes in that draft. For the next draft, number 31, Steve suggested that Alan might be an appropriate editor as he is working on the expression language, which is a key subject for the next draft. Simon would also like to own a draft and would consider this, but that he could not commit in the meeting. 3. Expression Language The group has previously discussed difficulties with forward/backward references in expressions. Mike observed that forward-referencing expressions can occur in a DFDL schema but could only be used during unparse. Discussing whether it is feasible to police this statically, Mike reckoned that while it may be difficult to analyze an expression to see whether it referred forward or not, this would probably be a decidable problem (eg, follow the dfdl:outputValue chain). Steve asked how we should specify the data type to be returned from an expression, there being two candidates: a) the XML Schema type of the DFDL property b) the 'resolved' data type of the DFDL property as needed by the parser Take dfdl:length as an example. The XML Schema type is 'string' because the field can accept numeric literals, expressions, regular expression, etc. But the parser will always want an integer. Agreed that an expression should return the 'resolved' data type. Steve asked whether, in the property descriptions, we should include the allowable return type from an expression. Mike believed that we should, as it may be distinct from the DFDL type for that field. So, the dfdl:length property description in the spec needs to say exactly what the options are - eg, "a literal integer, or an expression that resolves to an integer, or a regular expression that resolves to an integer". Using the XSD "maxOccurs" field as an example, which is normally an integer but may also be the token 'unbounded', Simon suggested that simply using the 'resolved' type may not be sufficient and that a processor will need to be aware that, in some cases, the result of an expression may not be the natural type. Mike concluded that we would need to specify both types as above and also any 'distinguished tokens'. Finally, should a DFDL engine automatically cast an expression result to the 'resolved' type, or instead strictly enforce the return type of the expression. The group felt the latter option to be preferable. (Alan Powell joined the meeting) 4. Nulls and defaults Steve would like to review his previous correspondance with Mike before discussing this further. It will be included in the agenda for next week's meeting. 5. Property Precedence Geoff and Steve have been preparing a proposal for precedence using a mind map. Steve will distribute this initial proposal for wider review. (Mike Beckerle and Suman Kalia left the meeting) 6. Entity references Alan has been looking at the use of XML entity references to more easily allow non-printable characters to be written into DFDL documents, and has distributed a proposal within IBM. There are some issues around this at the moment (need DTD to define entities, allowable characters in XML 1.0 docs). Alan is looking at these. This discussion in IBM had led to the concept of a mechanism to easily represent arbitrary whitespace, which is a common feature of text formats but which causes problems when modelling. Simon has experience with this concept and will send Steve a description of how PolarLake handle this.. Steve suggested we could handle this by allowing delimiters to be a list of allowable values, with the first used as a default on unparse. (We already have this idea for dfdl:nullValue). Simon observed that this could not handle arbitrary length whitespace. Steve said that we should have entities that cover that - like <WSP> and <OWSP> in IBM's WTX parser (the O meaning optional) - these are extremely useful. So then you could say things like (ignore incorrect entity syntax): dfdl:separator ="x0Dx0A x0D" meaning allow the separator to default to CRLF but allow LF on its own. However, Steve also pointed out that in the EDI data format the choice of delimiter comes from an expression, adding to the complexity, because the allowable value of the delimiter is then <value from expression> concatenated with <entity>. Is that supported by current spec?. Eg: dfdl:separator ="{..\delimiter} {..\delimiter}x0Dx0A {..\delimiter}x0D" Simon wondered if we could deal with this situation in a different way by perhaps handling it as 'delimiter padding' and having a DFDL option to allow/trim it. But he cautioned that we must avoid ambiguity - for example, to handle whitespace at the end of a delimiter which is followed by data which allows whitespace. Steve said that in that situation you have no choice but to explictly model the whitespace and not use the arbitrary entities. Geoff thought that if we did go for the trimming approach we may need to describe separate sets of rules for whitespace handling for the markup region and for the data region. Steve will take an action to come up with a proposal. 7. Other business Steve would like to discuss a model of ACORD AL3 length-prefixed data on the working group call, and will add an item to next week's agenda. Mike and Geoff have been corresponding on that. Within IBM, some changes have been proposed to Mike's UML model of DFDL. This will be circulated to the working group when IBM comments are complete. Meeting closed, 17:45 GMT Ian Parkinson WebSphere ESB Development Mail Point 211, Hursley Park, Hursley, Winchester, SO21 2JN, UK Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Agenda for today's DFDL call
by Steve Hanson 09 Jan '08

09 Jan '08

1) OGF 22 in Cambridge, MA Will we participate and if so who will go 2) Level set on spec drafts Mike circulated a proposal Then issued v030 (vNext in the proposal) Assign owners to spec work items in the vX+n drafts 3) Expression language Forward reference limitation - confirm What data type returned by an expression This reveals underlying question, namely a DFDL property has an XML schema type and a 'resolved' type for use by the parser. Eg: dfdl:length is a string in XML Schema terms, but must resolve to an integer ultimately. 4) Nulls and defaults, and whether we can drop useNullValueForDefault 5) ... Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: New call in details for Wed call
by Steve Hanson 08 Jan '08

08 Jan '08

Call will take place at usual time. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 08/01/2008 17:52 ----- Steve Hanson/UK/IBM 04/01/2008 13:10 To DFDL Working Group cc Subject New call in details for Wed call PARTICIPANT ACCESS INFORMATION: ------------------------------- PARTICIPANT PASSCODE: 8190865 TO JOIN A CONFERENCE 1) You must use a touch-tone phone to participate in an Instant Meeting Conference. 2) Dial the appropriate access number 3) Enter numeric passcode followed by a "#" INSTANT MEETING FEATURES The following Instant Meeting features are available: Press *1 private help menu. Press *3 roll call (private) Press *6 mute or un-mute individual line. Click http://www.mymeetings.com/audioconferencing/globalAccessFAQ.php for information on specific country access dialing. Country Toll Numbers Freephone/Toll Free Number AUSTRIA 43-1-928-9652 BELGIUM 32-2-400-6791 0800-4-9612 DENMARK 45-7014-0237 FRANCE LYON: 33-4-26-69-12-74 080-511-3725 FRANCE MARSEILLE: 33-4-86-06-00-74 080-511-3725 FRANCE PARIS: 33-1-70-70-81-58 080-511-3725 GERMANY 49-69-2222-20363 0800-100-6504 IRELAND 353-1-247-7768 ITALY 39-02-3600-3648 800-985-679 LUXEMBOURG 352-27-000-1312 NETHERLANDS 31-20-713-2741 0800-020-3237 SPAIN 34-91-414-1542 800-099-667 SWEDEN 46-8-566-10-780 SWITZERLAND 41-44-580-3334 0800-000-516 UNITED KINGDOM BIRMINGHAM: 44-121-210-9014 0800-279-9193 UNITED KINGDOM GLASGOW: 44-141-202-3214 0800-279-9193 UNITED KINGDOM LEEDS: 44-113-301-2114 0800-279-9193 UNITED KINGDOM LONDON: 44-20-7019-0808 0800-279-9193 UNITED KINGDOM MANCHESTER: 44-161-601-1414 0800-279-9193 USA 866-505-4412 Restrictions may exist when accessing freephone/toll free numbers using a mobile telephone. Country Freephone numbers cannot be used outside of the country listed. Please provide an International Country toll number to your participants as a back-up in the event of any Freephone restrictions being encountered. ------------------------------------------------------------------------------ Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Fw: [GRID-ANNOUNCE] Important OGF22 Information
by Steve Hanson 04 Jan '08

04 Jan '08

Suggest we discuss on the next call. Mike - you are best placed to attend - will you be able to? Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 04/01/2008 15:52 ----- "John Ehrig" <jehrig(a)inventures.com> Sent by: grid-announce-bounces(a)ogf.org 03/01/2008 18:16 To <grid-announce(a)ogf.org> cc Subject [GRID-ANNOUNCE] Important OGF22 Information OGF22, taking place February 25-29 in Cambridge, Massachusetts will feature a wide variety of new and exciting content. Please see the below important information and deadlines for this event. 1. Registration 2. Preliminary Program 3. Hotel Information 4. Student Scholarships 5. Group Chair Scheduling 6. Sponsorships 1. REGISTRATION Advanced registration for OGF22 will close January 11. Register now to save! http://www.ogf.org/OGF22/registration.php 2. PRELIMINARY PROGRAM * Keynote Sessions Charlie Catlett will make the Tuesday (2/26) morning keynote address. Charlie founded the Global Grid Forum (predecessor to OGF) and is currently the CIO of Argonne National Laboratory, Director of Argonne's Computing and Information Systems Division, and a Senior Fellow at the Argonne/University of Chicago Computation Institute. As CIO of Argonne, Charlie will bring unique perspectives on operating a world-class data center while providing leading-edge technology solutions to computational scientists. Other keynote speakers are in being lined up. Watch for further announcements as their availability is confirmed. * OGF Chartered Groups And BoFs - Monday, February 25 through Friday, February 29 Activities are scheduled throughout the entire week where many of OGF's chartered groups will meet to make further progress on their work. In addition, the latest hot topic in distributed computing - "Cloud Systems" and how it may or may not impact Grids will be tackled in a special BoF session. * OGF Specification Adoption Track and Demonstration Session - Monday, February 25 and Tuesday, February 26 Many OGF working groups have recently demonstrated interoperability of their specifications. These significant accomplishments illustrate how much of the work of OGF is shifting from developing specifications to seeing specifications being used and adopted into products. This multi-session track will feature contributions from developers and users showcasing their work implementing OGF specifications. In addition, live demonstrations will take place during the welcome reception on Monday evening. * Workshops OGF22 will feature a number of interesting workshops that will address a variety of important topics in Grid: a) Enterprise Adoption Workshop - Tuesday, February 26 This full day workshop - "Enabling the Next Generation IT Infrastructure" will focus on putting Grid in the context of the distributed computing infrastructure, including virtualization, service-oriented architecture, web services and next-generation networks. Emphasis will be on solving business problems and meeting business needs, with a focus on Applications On-boarding and Managing Infrastructure. Presentations and case studies are expected from representatives from the 451 Group, VMware, Intel, SAS, Platform Computing, DataSynapse and others that will show a variety of areas that can be addressed by Grid in practical and achievable ways. b) Financial Services Workshop - Wednesday, February 27 The financial services industry is one of the early adopters, and successful users, of Grid technology. As the technology matures, new (and old) issues emerge like expanding grids to support transactional workloads, data management and movement, and putting policies in place for charging and tracking usage. Presentations and case studies that describe the emerging issues the financial services industry is now facing in Grid will be featured in this half day workshop. c) Pharma/Life Sciences Workshop - Wednesday, February 27 The pharmaceutical industry and life sciences in general have begun to depend on in silico experiments as critical elements in the path to discovery. This has enabled exploration that is only limited by their ability to access sufficient computing resources and to store and manage the resultant mountain of data. Presentations and case studies from those that have turned to Grid-based solutions to meet these needs will be featured in this half day workshop. d) Data Management Workshop - Thursday, February 28 This full day workshop, co-hosted by OGF and the Storage Networking Industry Association (SNIA), will address data management challenges with a focus on file system and data movement. We will also explore potential synergies and common areas between OGF and SNIA. * Other Community Content - Monday, February 25 through Thursday, February 28 Sessions from the greater Grid community covering numerous topics and a variety of subjects will also be presented. These include workshops, tutorials, BoFs and presentations on marketing, education in emerging/developing countries, Grid Identity Providers, GridWay, and other topics. 3. HOTEL INFORMATION OGF has secured a block of sleeping rooms at the Hyatt Regency Cambridge (meeting location) at a special rate of $129 for single or double occupancy. Reservations must be made on or before February 8, 2008, to receive the discounted rate. Reservations can be made by calling the hotel directly (be sure to reference 'OGF22') or online via http://ogf.org/OGF22/lodging.php. 4. STUDENT SCHOLARSHIPS A limited number of student scholarships are available for qualifying students. Hosted by OGF and CoreGRID, the OGF22 student scholarship program provides a unique opportunity for students to participate in OGF program sessions and group sessions as well as network with distinguished community participants and leaders. Application details and eligibility requirements can be found at http://ogf.org/OGF22/students.php. 5. GROUP CHAIR SCHEDULING OGF Group Chair session scheduling will remain open until January 11. Session scheduling and document details along with a link to the online session request site can be accessed via http://ogf.org/OGF22/chairinfo.php. 6. SPONSORSHIP OPPORTUNITIES Sponsorship opportunities for OGF22 include Display Tables, Welcome Reception, Chair Appreciation Night, OGF Co-branded Promotional Item, Wireless, Lunches and Breaks, and Badge Lanyard. By sponsoring OGF22 your organization will gain greater visibility and recognition as a Grid thought leader in our global community. This will be our only OGF event in the USA for all of 2008, so please strongly consider sponsoring if you are interested in reaching this audience. The sponsorship prospectus provides additional information http://www.ogf.org/OGF22/sponsor.php See you in Cambridge! Register today http://www.ogf.org/OGF22/registration.php Regards, John Ehrig Open Grid Forum Enterprise and Marketing Program Manager 925 275-6677 www.ogf.org -- grid-announce mailing list grid-announce(a)ogf.org http://www.ogf.org/mailman/listinfo/grid-announce Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

New call in details for Wed call
by Steve Hanson 04 Jan '08

04 Jan '08

PARTICIPANT ACCESS INFORMATION: ------------------------------- PARTICIPANT PASSCODE: 8190865 TO JOIN A CONFERENCE 1) You must use a touch-tone phone to participate in an Instant Meeting Conference. 2) Dial the appropriate access number 3) Enter numeric passcode followed by a "#" INSTANT MEETING FEATURES The following Instant Meeting features are available: Press *1 private help menu. Press *3 roll call (private) Press *6 mute or un-mute individual line. Click http://www.mymeetings.com/audioconferencing/globalAccessFAQ.php for information on specific country access dialing. Country Toll Numbers Freephone/Toll Free Number AUSTRIA 43-1-928-9652 BELGIUM 32-2-400-6791 0800-4-9612 DENMARK 45-7014-0237 FRANCE LYON: 33-4-26-69-12-74 080-511-3725 FRANCE MARSEILLE: 33-4-86-06-00-74 080-511-3725 FRANCE PARIS: 33-1-70-70-81-58 080-511-3725 GERMANY 49-69-2222-20363 0800-100-6504 IRELAND 353-1-247-7768 ITALY 39-02-3600-3648 800-985-679 LUXEMBOURG 352-27-000-1312 NETHERLANDS 31-20-713-2741 0800-020-3237 SPAIN 34-91-414-1542 800-099-667 SWEDEN 46-8-566-10-780 SWITZERLAND 41-44-580-3334 0800-000-516 UNITED KINGDOM BIRMINGHAM: 44-121-210-9014 0800-279-9193 UNITED KINGDOM GLASGOW: 44-141-202-3214 0800-279-9193 UNITED KINGDOM LEEDS: 44-113-301-2114 0800-279-9193 UNITED KINGDOM LONDON: 44-20-7019-0808 0800-279-9193 UNITED KINGDOM MANCHESTER: 44-161-601-1414 0800-279-9193 USA 866-505-4412 Restrictions may exist when accessing freephone/toll free numbers using a mobile telephone. Country Freephone numbers cannot be used outside of the country listed. Please provide an International Country toll number to your participants as a back-up in the event of any Freephone restrictions being encountered. ------------------------------------------------------------------------------ Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

Today's call cancelled
by Steve Hanson 02 Jan '08

02 Jan '08

The existing conference call number has expired and I'm in the process of setting up a replacement. Regards, Steve Steve Hanson WebSphere Message Brokers Hursley, UK Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

1 0

DFDL - Affiliation changing for Mike Beckerle
by Mike Beckerle 12 Dec '07

12 Dec '07

DFDLers, I am leaving IBM to join a smaller company. The good news is that I will continue to be involved in DFDL-related work in my new capacity. I will forward contact info for my new position soon. In the mean time I can be reached at the address here: (encoded to avoid spam address scrapers) michael dot beckerle at alum dot mit dot edu ...mikeb

1 0

Can't make call today. Sorry
by Mike Beckerle 12 Dec '07

12 Dec '07

Sorry everyone. I cannot make the call today. ...mikeb

1 0

DFDL infoset and wildcards
by Mike Beckerle 07 Dec '07

07 Dec '07

Our Infoset section of the spec is silent about wildcards. Do we need an infoset information item that corresponds to things that match a wildcard? Or can wildcards just be element info items that lack [namespace] [dataType] and where the [schema] refers to the schema component that defines the wildcard, not to an element declaration. Comments? Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

2 1

OGF DFDL WG minutes 2007-12-05 call
by Mike Beckerle 06 Dec '07

06 Dec '07

OGF DFDL WG minutes 2007-12-05 call Suman Kalia, Simon Parker, Alan Powell, Mike Beckerle (who else? - was someone else on also) We discussed Output issues in the DFDL expression language: E.g.., an outputValueCalc for a field in the header of a data stream may contain information that requires you to know the rep, or length of the rep, of the whole data item. We concluded that this kind of thing can't be ruled out. Some formats just require buffering and are not streamable; however, implementations can vary on just how large a data item they're able to cope with here. Expression language section will include a subsection highlighting this issue and that implementations can vary here. Alan will update his expression language proposal and include this. Also suggested was a path length-from-to function that takes 2 path expressions and gives you the size of the represntation between them. (start of first, to last bit before start of 2nd). (I don't think we discussed a clear use case motivating this, but there may be one. We did discuss applications trying to fit data into limited size boxes, but the use case is not clear. Also note that all representation lengths are subject to change due to different starting alignments.) Nillable and Default: We also discussed the interaction of nillable and having a default. The sense of the group on the call is that we can restrict these so that if something is nillable it cannot also have a default value, and that the behavior of DFDL on output for a required element that is nillable but not in the logical data, is to create a null value. Everyone agreed that there is no need for a property useNullValueForDefault because this should always be the behavior. Mike will forward a proposal. ...mikeb Mike Beckerle STSM, Architect, Scalable Computing IBM Software Group Information Platform and Solutions Westborough, MA 01581 direct: voice and FAX 508-599-7148 assistant: Pam Riordan priordan(a)us.ibm.com 508-599-7046

2 2